184 Commits

Author	SHA1	Message	Date
David Dworken	9f32ae5d2e	Refactor deletion request creation to happen in a transaction to avoid 1-n DB queries	2023-10-15 10:29:47 -07:00
David Dworken	151dfb4009	Optimize query endpoint so that it applies all deletion requests in one SQL query rather than having a 1-n pattern	2023-10-15 10:19:30 -07:00
David Dworken	6fb6498515	Make deletion request processing happen in parallel to speed up query endpoint	2023-10-14 17:21:00 -07:00
David Dworken	218c70f5e7	Create background context for background update to ensure that the DB increment doesn't get terminated when the request finishes	2023-10-14 16:53:40 -07:00
David Dworken	4c1479b920	Add comment with example of creating index concurrently	2023-10-14 11:41:06 -07:00
David Dworken	56995aa255	Rephrase log message that causes sporadic test failures	2023-10-12 20:19:46 -07:00
David Dworken	3dce11e332	Fix linter error by specifying field names	2023-10-11 18:01:46 -07:00
David Dworken	7e6221ab24	Add datadog metric to keep track of 503 errors	2023-10-11 18:01:12 -07:00
David Dworken	f0dbcd6e3b	Add more thorough tests for updates	2023-10-09 21:41:30 -07:00
David Dworken	7ffe5fc1e2	Add logging for github rate limiting	2023-10-07 20:54:26 -07:00
David Dworken	d2788a9f31	Swap interface{} to any	2023-10-07 15:11:49 -07:00
David Dworken	2e793b2c10	Test refactoring to remove a function that can be replaced with testify	2023-10-01 18:21:11 -07:00
David Dworken	7ad1e2fb03	Upgrade to go 1.21 which offers fully reproducible builds (on top of SLSA's attestations)	2023-10-01 18:00:58 -07:00
David Dworken	ce883279f7	Follow up to f3764b2545: Update error code used for panics in unit tests	2023-09-30 22:22:31 -07:00
David Dworken	f9372dda75	Silence go-errcheck warning for fmt.Fprintf logging	2023-09-30 07:25:58 -07:00
David Dworken	f3764b2545	Update error code used for panics	2023-09-30 07:25:22 -07:00
David Dworken	2f5288f832	Merge branch 'panic-handling' into sergio/panic-guard	2023-09-30 07:22:12 -07:00
David Dworken	0b57034c39	Revert "Revert all commits since v0.223 to enable me to release a patch on top of v0.223" ... This reverts commit 68ed9f2d5d.	2023-09-29 18:21:23 -07:00
Sergio Moura	c079e13078	remove redundant test	2023-09-29 09:40:53 -04:00
Sergio Moura	6698cf526f	prevent server breaking when handlers panics, log even when panic happens, with tests	2023-09-29 09:38:06 -04:00
David Dworken	68ed9f2d5d	Revert all commits since v0.223 to enable me to release a patch on top of v0.223	2023-09-28 22:15:30 -07:00
David Dworken	4370820281	Skip storing user stats when running with sqlite, since the queries aren't compatible with sqlite	2023-09-28 22:15:16 -07:00
David Dworken	96e8e4f620	Add backend table to track daily/weekly active user stats	2023-09-28 22:14:55 -07:00
David Dworken	678ba25ecd	Update CreateIndices() function so that it works with sqlite	2023-09-28 22:14:44 -07:00
David Dworken	59119823f5	Fix bug in updateUsageData that caused it to not record usage data for newly created users	2023-09-28 22:13:31 -07:00
David Dworken	9fda54d4c2	Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace	2023-09-28 22:13:27 -07:00
David Dworken	c77d5a5424	Remove prints from server startup	2023-09-28 22:13:23 -07:00
David Dworken	6d5a86a8bb	Revert all commits since v0.223 to enable me to release a patch on top of v0.223	2023-09-28 21:49:37 -07:00
David Dworken	d0712f84c6	Skip storing user stats when running with sqlite, since the queries aren't compatible with sqlite	2023-09-26 20:49:21 -07:00
David Dworken	db4e31d997	Add backend table to track daily/weekly active user stats	2023-09-26 20:11:06 -07:00
David Dworken	3d0673e6e9	Update CreateIndices() function so that it works with sqlite	2023-09-24 18:49:40 -07:00
David Dworken	9b4af85f34	Fix bug in updateUsageData that caused it to not record usage data for newly created users	2023-09-24 16:14:05 -07:00
David Dworken	5121df5a8c	Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace	2023-09-24 16:05:01 -07:00
David Dworken	6c6245d962	Remove prints from server startup	2023-09-24 15:59:39 -07:00
David Dworken	1a144e4e70	Call TrackUsageData(true) to enable usage data tracking, which was accidentally disabled during server refactoring	2023-09-24 14:17:15 -07:00
David Dworken	8443292070	Augment IsOfflineError(err) so that it detects if the hishtory server is down, and will then treat all API errors as offline errors	2023-09-23 16:40:03 -07:00
David Dworken	4006eef1e1	Fix bug where requests missing the X-Hishtory-Version header wouldn't get dump/deletion requests (which caused tests to fail since the tests don't set this header)	2023-09-23 12:23:42 -07:00
David Dworken	b5db10d56a	Update submission endpoint to only include dump and deletion requests if the request is from a modern client, and update it to increment the relevant counters	2023-09-23 11:34:56 -07:00
David Dworken	47bdc966b2	Add GreaterThan method for comparing versions	2023-09-23 11:34:02 -07:00
David Dworken	6cfbec4823	Add struct + functions for parsing hishtory versions	2023-09-23 11:23:47 -07:00
David Dworken	eb85f9a293	Revert "Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt)" ... This reverts commit bbfda0be96.	2023-09-22 21:20:53 -07:00
David Dworken	bbfda0be96	Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt)	2023-09-22 19:20:51 -07:00
David Dworken	5bdbd9b262	Revert increased read_count requirement for deletion requests since deleting via encrypted-ID should be reliable	2023-09-22 19:19:06 -07:00
David Dworken	cc11916f3c	Create func to automatically create DB indexes rather than just documenting them in a comment that has to be manually executed	2023-09-22 19:11:43 -07:00
David Dworken	8c6443ed07	Fix incorrect column name	2023-09-22 18:59:37 -07:00
David Dworken	1d878195b2	Rework ff98a7907c to use the newly added EntryId column rather than deleting based on the start time	2023-09-22 14:03:41 -07:00
David Dworken	9b847c5e35	Further optimize client-server roundtrips by including deletion and dump requests in submit responses (follow up to 1e43de689f)	2023-09-22 13:49:29 -07:00
David Dworken	2a5a6d65c4	Roll-forward "Add preliminary support for persisting pre-saved history entries remotely" ... This rolls-forward commit 66916c27cb.	2023-09-22 13:13:46 -07:00
David Dworken	66916c27cb	Revert "Add preliminary support for persisting pre-saved history entries remotely" ... This reverts commit ff98a7907c. That commit is incomplete since it doesn't include support for the continous deletion of pre-saved history entries as soon as they finish running. Support for this will require a good bit more work/thought, so reverting for and keeping this code in the git history.	2023-09-21 12:39:20 -07:00
David Dworken	ff98a7907c	Add preliminary support for persisting pre-saved history entries remotely	2023-09-21 12:39:04 -07:00