David Dworken
cd53e9a40d
Add another filter check to remove integration test devices
2023-10-15 17:58:25 -07:00
David Dworken
690b969fe3
Use queryReason parameter to properly record query interactions
2023-10-15 17:45:38 -07:00
David Dworken
cd69973939
Update usage_data queries to filter out integration tests
2023-10-15 17:30:55 -07:00
David Dworken
24750066a7
Merge branch 'master' of github.com:ddworken/hishtory
2023-10-15 16:43:08 -07:00
David Dworken
5ff15303bf
exclude test devices from registration metrics
2023-10-15 16:42:59 -07:00
David Dworken
505daacb65
Add another TODO for cleaning up integration tests from the main DB
2023-10-15 14:54:01 -07:00
David Dworken
460d161c76
Update some usage calculations to exclude is_integration_test_device
2023-10-15 14:52:45 -07:00
David Dworken
fc865aef28
Add TODO
2023-10-15 14:50:17 -07:00
David Dworken
f3727dbeff
Wire through a flag so that we can track when installations come from tests, and delete those from the DB more aggressively
2023-10-15 13:09:48 -07:00
David Dworken
f6b3890f20
Fix incorrect comparisons for checking if a timestamp is zero-valued
2023-10-15 12:59:42 -07:00
David Dworken
67f2ed6de1
Fix transaction for creating deletion requests so that it actually uses the passed in tx
2023-10-15 11:49:36 -07:00
David Dworken
299617ba7e
Update error message to make it clearer when it shows up in test logs
2023-10-15 11:45:08 -07:00
David Dworken
cfefbb9b19
Improve DeleteMessagesFromBackend with better error checking and optimized queries for cases when MessageIdentifier is missing data
2023-10-15 11:43:29 -07:00
David Dworken
9f32ae5d2e
Refactor deletion request creation to happen in a transaction to avoid 1-n DB queries
2023-10-15 10:29:47 -07:00
David Dworken
151dfb4009
Optimize query endpoint so that it applies all deletion requests in one SQL query rather than having a 1-n pattern
2023-10-15 10:19:30 -07:00
David Dworken
211c9fb6b7
Update demo gif to show the most recent TUI
2023-10-14 18:43:04 -07:00
David Dworken
6fb6498515
Make deletion request processing happen in parallel to speed up query endpoint
2023-10-14 17:21:00 -07:00
David Dworken
218c70f5e7
Create background context for background update to ensure that the DB increment doesn't get terminated when the request finishes
2023-10-14 16:53:40 -07:00
David Dworken
4c1479b920
Add comment with example of creating index concurrently
2023-10-14 11:41:06 -07:00
David Dworken
56995aa255
Rephrase log message that causes sporadic test failures
2023-10-12 20:19:46 -07:00
David Dworken
3dce11e332
Fix linter error by specifying field names
2023-10-11 18:01:46 -07:00
David Dworken
7e6221ab24
Add datadog metric to keep track of 503 errors
2023-10-11 18:01:12 -07:00
David Dworken
f0dbcd6e3b
Add more thorough tests for updates
2023-10-09 21:41:30 -07:00
David Dworken
7ffe5fc1e2
Add logging for github rate limiting
2023-10-07 20:54:26 -07:00
David Dworken
d2788a9f31
Swap interface{} to any
2023-10-07 15:11:49 -07:00
David Dworken
2e793b2c10
Test refactoring to remove a function that can be replaced with testify
2023-10-01 18:21:11 -07:00
David Dworken
7ad1e2fb03
Upgrade to go 1.21 which offers fully reproducible builds (on top of SLSA's attestations)
2023-10-01 18:00:58 -07:00
David Dworken
ce883279f7
Follow up to f3764b2545: Update error code used for panics in unit tests
2023-09-30 22:22:31 -07:00
David Dworken
f9372dda75
Silence go-errcheck warning for fmt.Fprintf logging
2023-09-30 07:25:58 -07:00
David Dworken
f3764b2545
Update error code used for panics
2023-09-30 07:25:22 -07:00
David Dworken
2f5288f832
Merge branch 'panic-handling' into sergio/panic-guard
2023-09-30 07:22:12 -07:00
David Dworken
0b57034c39
Revert "Revert all commits since v0.223 to enable me to release a patch on top of v0.223"
...
This reverts commit 68ed9f2d5d
2023-09-29 18:21:23 -07:00
Sergio Moura
c079e13078
remove redundant test
2023-09-29 09:40:53 -04:00
Sergio Moura
6698cf526f
prevent server breaking when handlers panics, log even when panic happens, with tests
2023-09-29 09:38:06 -04:00
David Dworken
68ed9f2d5d
Revert all commits since v0.223 to enable me to release a patch on top of v0.223
2023-09-28 22:15:30 -07:00
David Dworken
4370820281
Skip storing user stats when running with sqlite, since the queries aren't compatible with sqlite
2023-09-28 22:15:16 -07:00
David Dworken
96e8e4f620
Add backend table to track daily/weekly active user stats
2023-09-28 22:14:55 -07:00
David Dworken
678ba25ecd
Update CreateIndices() function so that it works with sqlite
2023-09-28 22:14:44 -07:00
David Dworken
59119823f5
Fix bug in updateUsageData that caused it to not record usage data for newly created users
2023-09-28 22:13:31 -07:00
David Dworken
9fda54d4c2
Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace
2023-09-28 22:13:27 -07:00
David Dworken
c77d5a5424
Remove prints from server startup
2023-09-28 22:13:23 -07:00
David Dworken
6d5a86a8bb
Revert all commits since v0.223 to enable me to release a patch on top of v0.223
2023-09-28 21:49:37 -07:00
David Dworken
d0712f84c6
Skip storing user stats when running with sqlite, since the queries aren't compatible with sqlite
2023-09-26 20:49:21 -07:00
David Dworken
db4e31d997
Add backend table to track daily/weekly active user stats
2023-09-26 20:11:06 -07:00
David Dworken
3d0673e6e9
Update CreateIndices() function so that it works with sqlite
2023-09-24 18:49:40 -07:00
David Dworken
9b4af85f34
Fix bug in updateUsageData that caused it to not record usage data for newly created users
2023-09-24 16:14:05 -07:00
David Dworken
5121df5a8c
Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace
2023-09-24 16:05:01 -07:00
David Dworken
6c6245d962
Remove prints from server startup
2023-09-24 15:59:39 -07:00
David Dworken
1a144e4e70
Call TrackUsageData(true) to enable usage data tracking, which was accidentally disabled during server refactoring
2023-09-24 14:17:15 -07:00
David Dworken
8443292070
Augment IsOfflineError(err) so that it detects if the hishtory server is down, and will then treat all API errors as offline errors
2023-09-23 16:40:03 -07:00
David Dworken
4006eef1e1
Fix bug where requests missing the X-Hishtory-Version header wouldn't get dump/deletion requests (which caused tests to fail since the tests don't set this header)
2023-09-23 12:23:42 -07:00
David Dworken
b5db10d56a
Update submission endpoint to only include dump and deletion requests if the request is from a modern client, and update it to increment the relevant counters
2023-09-23 11:34:56 -07:00
David Dworken
47bdc966b2
Add GreaterThan method for comparing versions
2023-09-23 11:34:02 -07:00
David Dworken
6cfbec4823
Add struct + functions for parsing hishtory versions
2023-09-23 11:23:47 -07:00
David Dworken
eb85f9a293
Revert "Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt)"
...
This reverts commit bbfda0be96
2023-09-22 21:20:53 -07:00
David Dworken
bbfda0be96
Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt)
2023-09-22 19:20:51 -07:00
David Dworken
5bdbd9b262
Revert increased read_count requirement for deletion requests since deleting via encrypted-ID should be reliable
2023-09-22 19:19:06 -07:00
David Dworken
cc11916f3c
Create func to automatically create DB indexes rather than just documenting them in a comment that has to be manually executed
2023-09-22 19:11:43 -07:00
David Dworken
8c6443ed07
Fix incorrect column name
2023-09-22 18:59:37 -07:00
David Dworken
1d878195b2
Rework ff98a7907c to use the newly added EntryId column rather than deleting based on the start time
2023-09-22 14:03:41 -07:00
David Dworken
9b847c5e35
Further optimize client-server roundtrips by including deletion and dump requests in submit responses (follow up to 1e43de689f)
2023-09-22 13:49:29 -07:00
David Dworken
2a5a6d65c4
Roll-forward "Add preliminary support for persisting pre-saved history entries remotely"
...
This rolls-forward commit 66916c27cb
2023-09-22 13:13:46 -07:00
David Dworken
66916c27cb
Revert "Add preliminary support for persisting pre-saved history entries remotely"
...
This reverts commit ff98a7907c
2023-09-21 12:39:20 -07:00
David Dworken
ff98a7907c
Add preliminary support for persisting pre-saved history entries remotely
2023-09-21 12:39:04 -07:00
David Dworken
a3b865fa6b
Move all handlers into either api_handlers.go for client-exposed handlers, or debug_handlers.go for debug/admin functionality
2023-09-21 11:47:16 -07:00
David Dworken
9fc1140ab9
Swap json decoding to Decoder API for shorter/clearer code
2023-09-21 11:42:02 -07:00
David Dworken
1e43de689f
Optimize number of round-trip HTTP connections made by the client by having the submit handler return metadata about whether there are pending dump/deletion requests
...
For now, I'm still keeping the dedicated endpoints for those functionalities, but since most of the time there are no dump/deletion requests this should cut down the number of requests made by the client by 2/3.
2023-09-21 11:35:24 -07:00
David Dworken
f64362fbba
Fix metrics tag format so handler tags are properly exported to datadog
2023-09-17 22:46:31 -07:00
David Dworken
1ab61f1d7d
Fix missing err != nil branch
2023-09-17 13:07:38 -07:00
David Dworken
d94e1bfa22
Fix imports after moving backend/server code in 772aaf2453
2023-09-17 12:53:55 -07:00
David Dworken
772aaf2453
Move internal/ directory into backend/server since it is all server-specific internals
2023-09-17 12:51:21 -07:00
David Dworken
04a0fc40cd
Make errors from updateUsageData(...) crash when running in non-prod to ensure they're detected/handled before deployment
2023-09-17 12:03:42 -07:00
David Dworken
b733878878
Skip re-initializing tables when running in prod to make server startup faster
2023-09-15 17:51:14 -07:00
David Dworken
63741a0d14
Optimize docker build efficiency by running apk add early so that it is cached
2023-09-15 17:10:55 -07:00
David Dworken
1036e14c17
Add debug prints to debug server startup
2023-09-15 17:07:25 -07:00
David Dworken
a66ea1387d
Refactor server.go to remove two global variables
2023-09-13 21:47:06 -07:00
Sergio Moura
a8360efa67
revert main.ReleaseVersion changes
2023-09-12 15:55:41 -04:00
Sergio Moura
0d30011a33
break down release versions and fix server tests
2023-09-12 15:44:03 -04:00
Sergio Moura
60a0e20dd9
extract server object to its own package
2023-09-12 15:44:03 -04:00
Sergio Moura
02b1e8287d
isolate all server handlers into a single struct, without using global variables
2023-09-12 15:44:01 -04:00
David Dworken
50c74e5881
Rename a bunch of DB functions + add error checking for DB table creation (follow up to #104 )
2023-09-12 08:30:43 -07:00
Sergio Moura
eaa837d17d
fix check for empty array result (II)
2023-09-11 16:37:55 -04:00
Sergio Moura
ff2ac2194d
fix %w
2023-09-11 13:03:29 -04:00
Sergio Moura
f9ee8ed189
use DateOnly as local constant instead of using the imported time package
2023-09-11 10:15:42 -04:00
Sergio Moura
3c18f62d99
finishing removing direct DB instructions from http handlers
2023-09-08 10:58:01 -04:00
Sergio Moura
0d6aa081d8
remove direct db instructions from apiRegisterHandler and statsHandler
2023-09-08 10:58:01 -04:00
Sergio Moura
382f234c09
remove usage_data related queries from the main package into a database package
2023-09-08 10:58:01 -04:00
Sergio Moura
a3e6d5a940
docker improvements:
...
* use alpine for small docker images
* use docker compose to wait for the database to be ready
* use a single Dockerfile with parameters to determine the correct GOARCH
* update makefile to account for the new GOARCH parameter on the Dockerfile
* remove native-arch-Dockerfile
2023-09-07 11:06:32 -04:00
David Dworken
aa3c7fc41a
Follow up to #103 : assign err from fmt.Fprintf to _ to ensure go-errcheck doesn't flag it
2023-09-07 07:57:20 -07:00
David Dworken
68e3a813c9
Follow up to #103 : pull context from r.Context() when used rather than at the start of functions
2023-09-07 07:56:03 -07:00
Sergio Moura
e6d922709d
fix tests
2023-09-07 09:50:36 -04:00
Sergio Moura
37670230af
Merge remote-tracking branch 'origin/master' into sergio/handlers
2023-09-07 09:33:42 -04:00
Sergio Moura
589b99e500
do not use http.StatusNoContent
2023-09-07 08:34:21 -04:00
David Dworken
2a95553fdf
Fix install.py script for arm7 so installs work on raspberry pis
2023-09-06 21:35:28 -07:00
Sergio Moura
2b1ba7e3ba
use single context and always return a status to the client
...
api handlers do not need an extra context. http.Request already has a context that is being ignored, so we leverage it and stop creating a new one.
make the endpoints return http.StatusNoContent instead of just closing the connection from the client.
2023-09-06 11:37:14 -04:00
Sergio Moura
83ad8c7b1f
wrap errors with %w instead of using %v
2023-09-05 15:08:55 -04:00
David Dworken
f1aefdbf30
Fix download URL for arm7
2023-02-18 22:29:01 -08:00
David Dworken
27cd544524
Add support for arm7 for #63 to support raspberry pis
2023-02-18 22:00:39 -08:00
David Dworken
8dc5168c64
Set max open conns to 1 for test code to fix race conditions that lead to flaky tests
2023-02-13 20:50:33 -08:00
David Dworken
f502cbee1d
Fix memory leak caused by not closing resp.Body ( #62 ) and also fix the server to work when the DD integration is disabled
2022-12-23 17:17:44 -08:00
