Commit Graph

208 Commits

Author SHA1 Message Date
David Dworken
047ff97cfc
Add more thorough deep cleaning to delete entries for users that have more than 50 million entries, since these users are clearly misusing hishtory in some way that is causing DB load issues 2023-10-28 23:30:18 -07:00
David Dworken
f3c8ecc9e6
Update deep clean code to actually execute the delete now that I validated that it is reasonable 2023-10-28 15:06:58 -07:00
David Dworken
0c39dbf27c
Update deep clean code to be stricter and to fix a bug 2023-10-27 20:16:30 -07:00
David Dworken
0f99c7996e
Follow up to 083569be17: What if we just change the message we print on the server-side so that it doesn't trigger gotestsum? 2023-10-22 11:46:11 -07:00
David Dworken
53580d40e9
Remove TODOs for adding data to context that I think is actually fine to keep in the request as-is 2023-10-22 11:17:06 -07:00
David Dworken
083569be17
Move output prefix from 60d388c4c2 to the inner test 2023-10-22 11:07:59 -07:00
David Dworken
60d388c4c2
Add one more print similar to b6885cca98 that was missed in that commit 2023-10-22 00:19:34 -07:00
David Dworken
b6885cca98
Attempt to fix gotestsum skipping tests by adding a prefix so that test output doesn't begin with panic: 2023-10-21 17:03:04 -07:00
David Dworken
50c6f5e061
Skip tests that cause gotestsum to skip re-running 2023-10-21 16:59:42 -07:00
David Dworken
a937b35e54
Avoid returning DB error when deleted message identifiers don't have a device ID 2023-10-18 18:59:12 -07:00
David Dworken
9a4a2a18aa
Update TODOs 2023-10-15 17:59:11 -07:00
David Dworken
cd53e9a40d
Add another filter check to remove integration test devices 2023-10-15 17:58:25 -07:00
David Dworken
690b969fe3
Use queryReason parameter to properly record query interactions 2023-10-15 17:45:38 -07:00
David Dworken
cd69973939
Update usage_data queries to filter out integration tests 2023-10-15 17:30:55 -07:00
David Dworken
24750066a7 Merge branch 'master' of github.com:ddworken/hishtory 2023-10-15 16:43:08 -07:00
David Dworken
5ff15303bf exclude test devices from registration metrics 2023-10-15 16:42:59 -07:00
David Dworken
505daacb65
Add another TODO for cleaning up integration tests from the main DB 2023-10-15 14:54:01 -07:00
David Dworken
460d161c76
Update some usage calculations to exclude is_integration_test_device 2023-10-15 14:52:45 -07:00
David Dworken
fc865aef28
Add TODO 2023-10-15 14:50:17 -07:00
David Dworken
f3727dbeff
Wire through a flag so that we can track when installations come from tests, and delete those from the DB more aggressively 2023-10-15 13:09:48 -07:00
David Dworken
f6b3890f20
Fix incorrect comparisons for checking if a timestamp is zero-valued 2023-10-15 12:59:42 -07:00
David Dworken
67f2ed6de1
Fix transaction for creating deletion requests so that it actually uses the passed in tx 2023-10-15 11:49:36 -07:00
David Dworken
299617ba7e
Update error message to make it clearer when it shows up in test logs 2023-10-15 11:45:08 -07:00
David Dworken
cfefbb9b19
Improve DeleteMessagesFromBackend with better error checking and optimized queries for cases when MessageIdentifier is missing data 2023-10-15 11:43:29 -07:00
David Dworken
9f32ae5d2e
Refactor deletion request creation to happen in a transaction to avoid 1-n DB queries 2023-10-15 10:29:47 -07:00
David Dworken
151dfb4009
Optimize query endpoint so that it applies all deletion requests in one SQL query rather than having a 1-n pattern 2023-10-15 10:19:30 -07:00
David Dworken
6fb6498515
Make deletion request processing happen in parallel to speed up query endpoint 2023-10-14 17:21:00 -07:00
David Dworken
218c70f5e7
Create background context for background update to ensure that the DB increment doesn't get terminated when the request finishes 2023-10-14 16:53:40 -07:00
David Dworken
4c1479b920
Add comment with example of creating index concurrently 2023-10-14 11:41:06 -07:00
David Dworken
56995aa255
Rephrase log message that causes sporadic test failures 2023-10-12 20:19:46 -07:00
David Dworken
3dce11e332
Fix linter error by specifying field names 2023-10-11 18:01:46 -07:00
David Dworken
7e6221ab24
Add datadog metric to keep track of 503 errors 2023-10-11 18:01:12 -07:00
David Dworken
f0dbcd6e3b
Add more thorough tests for updates 2023-10-09 21:41:30 -07:00
David Dworken
7ffe5fc1e2
Add logging for github rate limiting 2023-10-07 20:54:26 -07:00
David Dworken
d2788a9f31
Swap interface{} to any 2023-10-07 15:11:49 -07:00
David Dworken
2e793b2c10
Test refactoring to remove a function that can be replaced with testify 2023-10-01 18:21:11 -07:00
David Dworken
7ad1e2fb03
Upgrade to go 1.21 which offers fully reproducible builds (on top of SLSA's attestations) 2023-10-01 18:00:58 -07:00
David Dworken
ce883279f7
Follow up to f3764b2545: Update error code used for panics in unit tests 2023-09-30 22:22:31 -07:00
David Dworken
f9372dda75
Silence go-errcheck warning for fmt.Fprintf logging 2023-09-30 07:25:58 -07:00
David Dworken
f3764b2545
Update error code used for panics 2023-09-30 07:25:22 -07:00
David Dworken
2f5288f832
Merge branch 'panic-handling' into sergio/panic-guard 2023-09-30 07:22:12 -07:00
David Dworken
0b57034c39
Revert "Revert all commits since v0.223 to enable me to release a patch on top of v0.223"
This reverts commit 68ed9f2d5d.
2023-09-29 18:21:23 -07:00
Sergio Moura
c079e13078 remove redundant test 2023-09-29 09:40:53 -04:00
Sergio Moura
6698cf526f prevent server breaking when handlers panics, log even when panic happens, with tests 2023-09-29 09:38:06 -04:00
David Dworken
68ed9f2d5d
Revert all commits since v0.223 to enable me to release a patch on top of v0.223 2023-09-28 22:15:30 -07:00
David Dworken
4370820281
Skip storing user stats when running with sqlite, since the queries aren't compatible with sqlite 2023-09-28 22:15:16 -07:00
David Dworken
96e8e4f620
Add backend table to track daily/weekly active user stats 2023-09-28 22:14:55 -07:00
David Dworken
678ba25ecd
Update CreateIndices() function so that it works with sqlite 2023-09-28 22:14:44 -07:00
David Dworken
59119823f5
Fix bug in updateUsageData that caused it to not record usage data for newly created users 2023-09-28 22:13:31 -07:00
David Dworken
9fda54d4c2
Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace 2023-09-28 22:13:27 -07:00