David Dworken
1e43de689f
Optimize number of round-trip HTTP connections made by the client by having the submit handler return metadata about whether there are pending dump/deletion requests
...
For now, I'm still keeping the dedicated endpoints for those functionalities, but since most of the time there are no dump/deletion requests this should cut down the number of requests made by the client by 2/3.
2023-09-21 11:35:24 -07:00
David Dworken
f64362fbba
Fix metrics tag format so handler tags are properly exported to datadog
2023-09-17 22:46:31 -07:00
David Dworken
1ab61f1d7d
Fix missing err != nil branch
2023-09-17 13:07:38 -07:00
David Dworken
d94e1bfa22
Fix imports after moving backend/server code in 772aaf2453
2023-09-17 12:53:55 -07:00
David Dworken
772aaf2453
Move internal/ directory into backend/server since it is all server-specific internals
2023-09-17 12:51:21 -07:00
David Dworken
04a0fc40cd
Make errors from updateUsageData(...) crash when running in non-prod to ensure they're detected/handled before deployment
2023-09-17 12:03:42 -07:00
David Dworken
b733878878
Skip re-initializing tables when running in prod to make server startup faster
2023-09-15 17:51:14 -07:00
David Dworken
63741a0d14
Optimize docker build efficiency by running apk add early so that it is cached
2023-09-15 17:10:55 -07:00
David Dworken
1036e14c17
Add debug prints to debug server startup
2023-09-15 17:07:25 -07:00
David Dworken
a66ea1387d
Refactor server.go to remove two global variables
2023-09-13 21:47:06 -07:00
Sergio Moura
a8360efa67
revert main.ReleaseVersion changes
2023-09-12 15:55:41 -04:00
Sergio Moura
0d30011a33
break down release versions and fix server tests
2023-09-12 15:44:03 -04:00
Sergio Moura
60a0e20dd9
extract server object to its own package
2023-09-12 15:44:03 -04:00
Sergio Moura
02b1e8287d
isolate all server handlers into a single struct, without using global variables
2023-09-12 15:44:01 -04:00
David Dworken
50c74e5881
Rename a bunch of DB functions + add error checking for DB table creation (follow up to #104 )
2023-09-12 08:30:43 -07:00
Sergio Moura
eaa837d17d
fix check for empty array result (II)
2023-09-11 16:37:55 -04:00
Sergio Moura
ff2ac2194d
fix %w
2023-09-11 13:03:29 -04:00
Sergio Moura
f9ee8ed189
use DateOnly as local constant instead of using the imported time package
2023-09-11 10:15:42 -04:00
Sergio Moura
3c18f62d99
finishing removing direct DB instructions from http handlers
2023-09-08 10:58:01 -04:00
Sergio Moura
0d6aa081d8
remove direct db instructions from apiRegisterHandler and statsHandler
2023-09-08 10:58:01 -04:00
Sergio Moura
382f234c09
remove usage_data related queries from the main package into a database package
2023-09-08 10:58:01 -04:00
Sergio Moura
a3e6d5a940
docker improvements:
...
* use alpine for small docker images
* use docker compose to wait for the database to be ready
* use a single Dockerfile with parameters to determine the correct GOARCH
* update makefile to account for the new GOARCH parameter on the Dockerfile
* remove native-arch-Dockerfile
2023-09-07 11:06:32 -04:00
David Dworken
aa3c7fc41a
Follow up to #103 : assign err from fmt.Fprintf to _ to ensure go-errcheck doesn't flag it
2023-09-07 07:57:20 -07:00
David Dworken
68e3a813c9
Follow up to #103 : pull context from r.Context() when used rather than at the start of functions
2023-09-07 07:56:03 -07:00
Sergio Moura
e6d922709d
fix tests
2023-09-07 09:50:36 -04:00
Sergio Moura
37670230af
Merge remote-tracking branch 'origin/master' into sergio/handlers
2023-09-07 09:33:42 -04:00
Sergio Moura
589b99e500
do not use http.StatusNoContent
2023-09-07 08:34:21 -04:00
David Dworken
2a95553fdf
Fix install.py script for arm7 so installs work on raspberry pis
2023-09-06 21:35:28 -07:00
Sergio Moura
2b1ba7e3ba
use single context and always return a status to the client
...
api handlers do not need an extra context. http.Request already has a context that is being ignored, so we leverage it and stop creating a new one.
make the endpoints return http.StatusNoContent instead of just closing the connection from the client.
2023-09-06 11:37:14 -04:00
Sergio Moura
83ad8c7b1f
wrap errors with %w instead of using %v
2023-09-05 15:08:55 -04:00
David Dworken
f1aefdbf30
Fix download URL for arm7
2023-02-18 22:29:01 -08:00
David Dworken
27cd544524
Add support for arm7 for #63 to support raspberry pis
2023-02-18 22:00:39 -08:00
David Dworken
8dc5168c64
Set max open conns to 1 for test code to fix race conditions that lead to flaky tests
2023-02-13 20:50:33 -08:00
David Dworken
f502cbee1d
Fix memory leak caused by not closing resp.Body ( #62 ) and also fix the server to work when the DD integration is disabled
2022-12-23 17:17:44 -08:00
David Dworken
06a4bf0bcb
Add code to deep clean the DB to remove entries from people with 1 device that haven't been active in at least 90 days
2022-12-17 21:27:00 -08:00
David Dworken
956377d472
linux arm is called aarch64, see #48
2022-12-12 19:45:54 -08:00
David Dworken
83667a52f4
Fix #51 by changing if --> elif
2022-12-12 19:44:02 -08:00
David Dworken
7c86b812bf
Add support for linux arm64 for #48
2022-12-11 20:39:45 -08:00
David Dworken
857e423e10
Fix linter errors + some general clean up
2022-12-11 20:31:50 -08:00
David Dworken
9ed325e0a5
Add support for limiting the number of registrations to fix #46
2022-12-11 19:42:51 -08:00
David Dworken
35444bf56e
Swap ioutil to non-deprecated alternatives + clean up pre-commit errors
2022-11-27 11:59:06 -08:00
David Dworken
3985010a17
Add test for the healthcheck endpoint
2022-11-26 22:53:14 -08:00
David Dworken
22330f3ee1
Ensure that tests assert a lack of leaked connections
2022-11-26 22:40:43 -08:00
David Dworken
f77eb38618
Rename misleading var
2022-11-26 22:16:11 -08:00
David Dworken
2ba535b650
Call Close() on sql.Rows to prevent connection leak
2022-11-26 22:15:16 -08:00
David Dworken
4e84f49de9
Revert "Move usage data into tx to optimize updating usage data"
...
This reverts commit aa501262e6
2022-11-26 18:59:32 -08:00
David Dworken
aa501262e6
Move usage data into tx to optimize updating usage data
2022-11-26 18:46:20 -08:00
David Dworken
865ce06b95
Optimize healthcheck handler to avoid unnecessary ping in prod + add prod-only config for connection reuse
2022-11-26 18:33:54 -08:00
David Dworken
a98bff0db8
Optimize query latency by moving the read count incrementing to a background task
2022-11-26 12:10:18 -08:00
David Dworken
c603537137
Optimize the apiSubmit function to ensure that it doesn't lead to errors from large insertions
2022-11-26 10:31:43 -08:00
David Dworken
8d87110405
Optimize the healthcheck analyzer to avoid a full table scan
2022-11-25 20:35:17 -08:00
David Dworken
1ce20157c7
Add datadog integration for monitoring the server
2022-11-25 20:04:40 -08:00
David Dworken
8dd9c1d9e4
Create the feedback table in pg
2022-11-20 00:12:41 -08:00
David Dworken
97d7c7e310
Add last registration date to internal status page
2022-11-19 09:41:40 -08:00
David Dworken
0bd9249df3
Update the healthcheck endpoint so it also writes to the DB to detect future cases of #38
2022-11-17 23:43:25 -08:00
David Dworken
d4efc62df1
Add client-side code to ask for feedback when uninstalling
2022-11-16 21:54:05 -08:00
David Dworken
48c77d88f7
Add endpoint to handle feedback reports
2022-11-16 21:42:07 -08:00
David Dworken
78d33cf437
Make the healthcheck endpoint work for non-global production instance usages to fix #35
2022-11-16 20:58:19 -08:00
David Dworken
fd3d932e9d
Document that the server can use sqlite
2022-11-16 20:54:47 -08:00
David Dworken
4aacd8b5e2
Merge pull request #31 from matthewcmead/sqlite_for_server
...
allow for sqlite backend database for low-profile self-hosting deploy…
2022-11-16 20:50:34 -08:00
David Dworken
35208680d8
Improve debug logs in server.go
2022-11-16 20:26:44 -08:00
m
77af69de86
allow for sqlite backend database for low-profile self-hosting deployments
2022-11-14 11:35:05 -05:00
David Dworken
3e0ed67518
Serve a HTML redirect
2022-11-06 14:59:00 -08:00
David Dworken
9cc485f3b1
Remove the redirect since it breaks the install.py
2022-11-06 14:43:49 -08:00
David Dworken
7ceb303caf
Delete the landing page and instead serve a redirect to the github repo
2022-11-06 14:33:28 -08:00
David Dworken
c2802f40dc
Use wait-for instead of a sleep for the dockerfile for self hosting
2022-11-05 13:46:08 -07:00
David Dworken
9ddae45b03
Add fully tested instructions for self hosting
2022-11-05 13:30:37 -07:00
David Dworken
36f776053b
Add docker-compose file for self hosting to fix #7
2022-11-04 23:41:56 -07:00
David Dworken
846f256fa4
Support a TMPDIR variable in case /tmp/ is mounted noexec
2022-11-03 18:04:16 -07:00
David Dworken
436d432065
Add warning message for the buggy slow version of hishtory
2022-11-02 20:52:37 -07:00
David Dworken
cc2a6046d3
Update demo gif to show search atoms
2022-11-01 22:39:54 -07:00
David Dworken
31ef7a2fdd
Better timing for the demo gif
2022-11-01 22:33:48 -07:00
David Dworken
24be30d1e1
Slow down gif
2022-11-01 19:13:47 -07:00
David Dworken
67bb2e979f
Add demo gif showing the control-r integration
2022-11-01 15:39:46 -07:00
David Dworken
4f708aa58c
Remove hard coded pg password and pull the new rotated password from an environment variable
2022-11-01 12:14:20 -07:00
David Dworken
19fbeda539
Set a correct version number for builds containing slsa-verifier <1.3.1
2022-11-01 11:16:23 -07:00
David Dworken
2efa93762a
Another attempt at getting github actions to pass
2022-11-01 11:03:49 -07:00
David Dworken
d2558c8fb5
Skip slsa updates only for the orphaned versions
2022-11-01 10:51:08 -07:00
David Dworken
2183a63f8f
Add an endpoint that is hit to determine if slsa is currently broken
2022-10-31 17:32:27 -07:00
David Dworken
cdc5421a7b
Move testutils to a separate package so as to move test-only code out of the main binary
2022-10-27 21:53:47 -07:00
David Dworken
b4936d588c
Add a very basic healthcheck handler to the backend
2022-10-23 17:54:07 -07:00
David Dworken
70c02bf0ed
Remove basic auth since that is now done by the LB
2022-10-22 11:08:33 -07:00
David Dworken
6b2e9059a4
Optimize the submit endpoint to only use 1 query and 1 insert rather than N
2022-10-18 14:52:35 -07:00
David Dworken
87d3dc4d97
Log the actual remote addr in log lines
2022-10-18 14:35:03 -07:00
David Dworken
ad809efef6
Migrate hishtory from k8s to docker compose
2022-10-18 11:53:35 -07:00
David Dworken
ab95b16ea2
Update comment
2022-10-17 20:38:40 -07:00
David Dworken
0da8021ab7
HTML escape the forced banner to prevent XSS (though this domain has no cookies so there would have been very limited harm from an XSS)
2022-10-15 09:58:47 -07:00
David Dworken
a40e44c985
Add explanation of why expectedPasswordHash is ok
2022-10-15 09:26:54 -07:00
David Dworken
38c185c9f9
Add environ variable to configure the postgres DB
2022-10-15 09:21:10 -07:00
David Dworken
68416f9d0f
Better string formatting for the internal analytics endpoint
2022-10-04 12:58:00 -07:00
David Dworken
9d95b4ca70
Add comma separation in stats page
2022-10-02 20:42:10 -07:00
David Dworken
a85315d366
Log hishtory version in internal analytics
2022-10-02 20:39:52 -07:00
David Dworken
b7c64b61c8
Always check gorm interactions for errors
2022-10-02 19:41:00 -07:00
David Dworken
757ebb9547
Prompt people if they run hishtory init and already have a bunch of entries + fix tests + add TODOs + add hishtory version to requests
2022-10-01 09:50:06 -07:00
David Dworken
18ddbf2ca9
Track queries in usage data + better formatting in the internal usage page
2022-09-30 23:38:35 -07:00
David Dworken
98a4f002fa
Get RemoteAddr from X-Real-Ip header
2022-09-29 23:51:45 -07:00
David Dworken
f0a3caed1c
Add an internal only handler to expose basic usage stats via HTTP and keep a per-device counter of number of history entries for my own vanity stats
2022-09-29 23:32:13 -07:00
David Dworken
a108268925
Add TODOs
2022-09-27 22:15:31 -07:00
David Dworken
5fc3cf3ebc
Increase speed
2022-09-24 16:14:05 -07:00
David Dworken
aa4a7310dd
Add hishtory.svg from asciinema
2022-09-24 16:12:02 -07:00
