Sergio Moura
02b1e8287d
isolate all server handlers into a single struct, without using global variables
2023-09-12 15:44:01 -04:00
David Dworken
50c74e5881
Rename a bunch of DB functions + add error checking for DB table creation (follow up to #104 )
2023-09-12 08:30:43 -07:00
Sergio Moura
eaa837d17d
fix check for empty array result (II)
2023-09-11 16:37:55 -04:00
Sergio Moura
ff2ac2194d
fix %w
2023-09-11 13:03:29 -04:00
Sergio Moura
f9ee8ed189
use DateOnly as local constant instead of using the imported time package
2023-09-11 10:15:42 -04:00
Sergio Moura
3c18f62d99
finishing removing direct DB instructions from http handlers
2023-09-08 10:58:01 -04:00
Sergio Moura
0d6aa081d8
remove direct db instructions from apiRegisterHandler and statsHandler
2023-09-08 10:58:01 -04:00
Sergio Moura
382f234c09
remove usage_data related queries from the main package into a database package
2023-09-08 10:58:01 -04:00
Sergio Moura
a3e6d5a940
docker improvements:
...
* use alpine for small docker images
* use docker compose to wait for the database to be ready
* use a single Dockerfile with parameters to determine the correct GOARCH
* update makefile to account for the new GOARCH parameter on the Dockerfile
* remove native-arch-Dockerfile
2023-09-07 11:06:32 -04:00
David Dworken
aa3c7fc41a
Follow up to #103 : assign err from fmt.Fprintf to _ to ensure go-errcheck doesn't flag it
2023-09-07 07:57:20 -07:00
David Dworken
68e3a813c9
Follow up to #103 : pull context from r.Context() when used rather than at the start of functions
2023-09-07 07:56:03 -07:00
Sergio Moura
e6d922709d
fix tests
2023-09-07 09:50:36 -04:00
Sergio Moura
37670230af
Merge remote-tracking branch 'origin/master' into sergio/handlers
2023-09-07 09:33:42 -04:00
Sergio Moura
589b99e500
do not use http.StatusNoContent
2023-09-07 08:34:21 -04:00
Sergio Moura
2b1ba7e3ba
use single context and always return a status to the client
...
api handlers do not need an extra context. http.Request already has a context that is being ignored, so we leverage it and stop creating a new one.
make the endpoints return http.StatusNoContent instead of just closing the connection from the client.
2023-09-06 11:37:14 -04:00
Sergio Moura
83ad8c7b1f
wrap errors with %w instead of using %v
2023-09-05 15:08:55 -04:00
David Dworken
f1aefdbf30
Fix download URL for arm7
2023-02-18 22:29:01 -08:00
David Dworken
27cd544524
Add support for arm7 for #63 to support raspberry pis
2023-02-18 22:00:39 -08:00
David Dworken
8dc5168c64
Set max open conns to 1 for test code to fix race conditions that lead to flaky tests
2023-02-13 20:50:33 -08:00
David Dworken
f502cbee1d
Fix memory leak caused by not closing resp.Body ( #62 ) and also fix the server to work when the DD integration is disabled
2022-12-23 17:17:44 -08:00
David Dworken
06a4bf0bcb
Add code to deep clean the DB to remove entries from people with 1 device that haven't been active in at least 90 days
2022-12-17 21:27:00 -08:00
David Dworken
7c86b812bf
Add support for linux arm64 for #48
2022-12-11 20:39:45 -08:00
David Dworken
857e423e10
Fix linter errors + some general clean up
2022-12-11 20:31:50 -08:00
David Dworken
9ed325e0a5
Add support for limiting the number of registrations to fix #46
2022-12-11 19:42:51 -08:00
David Dworken
35444bf56e
Swap ioutil to non-deprecated alternatives + clean up pre-commit errors
2022-11-27 11:59:06 -08:00
David Dworken
3985010a17
Add test for the healthcheck endpoint
2022-11-26 22:53:14 -08:00
David Dworken
22330f3ee1
Ensure that tests assert a lack of leaked connections
2022-11-26 22:40:43 -08:00
David Dworken
f77eb38618
Rename misleading var
2022-11-26 22:16:11 -08:00
David Dworken
2ba535b650
Call Close() on sql.Rows to prevent connection leak
2022-11-26 22:15:16 -08:00
David Dworken
4e84f49de9
Revert "Move usage data into tx to optimize updating usage data"
...
This reverts commit aa501262e6
2022-11-26 18:59:32 -08:00
David Dworken
aa501262e6
Move usage data into tx to optimize updating usage data
2022-11-26 18:46:20 -08:00
David Dworken
865ce06b95
Optimize healthcheck handler to avoid unnecessary ping in prod + add prod-only config for connection reuse
2022-11-26 18:33:54 -08:00
David Dworken
a98bff0db8
Optimize query latency by moving the read count incrementing to a background task
2022-11-26 12:10:18 -08:00
David Dworken
c603537137
Optimize the apiSubmit function to ensure that it doesn't lead to errors from large insertions
2022-11-26 10:31:43 -08:00
David Dworken
8d87110405
Optimize the healthcheck analyzer to avoid a full table scan
2022-11-25 20:35:17 -08:00
David Dworken
1ce20157c7
Add datadog integration for monitoring the server
2022-11-25 20:04:40 -08:00
David Dworken
8dd9c1d9e4
Create the feedback table in pg
2022-11-20 00:12:41 -08:00
David Dworken
97d7c7e310
Add last registration date to internal status page
2022-11-19 09:41:40 -08:00
David Dworken
0bd9249df3
Update the healthcheck endpoint so it also writes to the DB to detect future cases of #38
2022-11-17 23:43:25 -08:00
David Dworken
d4efc62df1
Add client-side code to ask for feedback when uninstalling
2022-11-16 21:54:05 -08:00
David Dworken
48c77d88f7
Add endpoint to handle feedback reports
2022-11-16 21:42:07 -08:00
David Dworken
78d33cf437
Make the healthcheck endpoint work for non-global production instance usages to fix #35
2022-11-16 20:58:19 -08:00
David Dworken
fd3d932e9d
Document that the server can use sqlite
2022-11-16 20:54:47 -08:00
David Dworken
4aacd8b5e2
Merge pull request #31 from matthewcmead/sqlite_for_server
...
allow for sqlite backend database for low-profile self-hosting deploy…
2022-11-16 20:50:34 -08:00
David Dworken
35208680d8
Improve debug logs in server.go
2022-11-16 20:26:44 -08:00
m
77af69de86
allow for sqlite backend database for low-profile self-hosting deployments
2022-11-14 11:35:05 -05:00
David Dworken
c2802f40dc
Use wait-for instead of a sleep for the dockerfile for self hosting
2022-11-05 13:46:08 -07:00
David Dworken
9ddae45b03
Add fully tested instructions for self hosting
2022-11-05 13:30:37 -07:00
David Dworken
36f776053b
Add docker-compose file for self hosting to fix #7
2022-11-04 23:41:56 -07:00
David Dworken
436d432065
Add warning message for the buggy slow version of hishtory
2022-11-02 20:52:37 -07:00
David Dworken
4f708aa58c
Remove hard coded pg password and pull the new rotated password from an environment variable
2022-11-01 12:14:20 -07:00
David Dworken
19fbeda539
Set a correct version number for builds containing slsa-verifier <1.3.1
2022-11-01 11:16:23 -07:00
David Dworken
2efa93762a
Another attempt at getting github actions to pass
2022-11-01 11:03:49 -07:00
David Dworken
d2558c8fb5
Skip slsa updates only for the orphaned versions
2022-11-01 10:51:08 -07:00
David Dworken
2183a63f8f
Add an endpoint that is hit to determine if slsa is currently broken
2022-10-31 17:32:27 -07:00
David Dworken
cdc5421a7b
Move testutils to a separate package so as to move test-only code out of the main binary
2022-10-27 21:53:47 -07:00
David Dworken
b4936d588c
Add a very basic healthcheck handler to the backend
2022-10-23 17:54:07 -07:00
David Dworken
70c02bf0ed
Remove basic auth since that is now done by the LB
2022-10-22 11:08:33 -07:00
David Dworken
6b2e9059a4
Optimize the submit endpoint to only use 1 query and 1 insert rather than N
2022-10-18 14:52:35 -07:00
David Dworken
87d3dc4d97
Log the actual remote addr in log lines
2022-10-18 14:35:03 -07:00
David Dworken
ad809efef6
Migrate hishtory from k8s to docker compose
2022-10-18 11:53:35 -07:00
David Dworken
ab95b16ea2
Update comment
2022-10-17 20:38:40 -07:00
David Dworken
0da8021ab7
HTML escape the forced banner to prevent XSS (though this domain has no cookies so there would have been very limited harm from an XSS)
2022-10-15 09:58:47 -07:00
David Dworken
a40e44c985
Add explanation of why expectedPasswordHash is ok
2022-10-15 09:26:54 -07:00
David Dworken
38c185c9f9
Add environ variable to configure the postgres DB
2022-10-15 09:21:10 -07:00
David Dworken
68416f9d0f
Better string formatting for the internal analytics endpoint
2022-10-04 12:58:00 -07:00
David Dworken
9d95b4ca70
Add comma separation in stats page
2022-10-02 20:42:10 -07:00
David Dworken
a85315d366
Log hishtory version in internal analytics
2022-10-02 20:39:52 -07:00
David Dworken
b7c64b61c8
Always check gorm interactions for errors
2022-10-02 19:41:00 -07:00
David Dworken
757ebb9547
Prompt people if they run hishtory init and already have a bunch of entries + fix tests + add TODOs + add hishtory version to requests
2022-10-01 09:50:06 -07:00
David Dworken
18ddbf2ca9
Track queries in usage data + better formatting in the internal usage page
2022-09-30 23:38:35 -07:00
David Dworken
98a4f002fa
Get RemoteAddr from X-Real-Ip header
2022-09-29 23:51:45 -07:00
David Dworken
f0a3caed1c
Add an internal only handler to expose basic usage stats via HTTP and keep a per-device counter of number of history entries for my own vanity stats
2022-09-29 23:32:13 -07:00
David Dworken
a108268925
Add TODOs
2022-09-27 22:15:31 -07:00
David Dworken
c18c2502cc
Add a complex test with many users
2022-09-22 21:45:20 -07:00
David Dworken
c6d4f1ef68
Fix missing newline in zshrc, add test for hishtory redact prompting, and tag TODOs with plans
2022-09-22 18:22:06 -07:00
David Dworken
ee6680f571
Remove debugging information and give up on that test, and add work around for weird zsh bug with importing
2022-09-22 18:09:51 -07:00
David Dworken
6db8044bec
Add server tests for redaction
2022-09-21 21:46:46 -07:00
David Dworken
27157e675d
Add TODOs
2022-09-21 19:59:31 -07:00
David Dworken
414d8926f6
Fix incorrect go version in dockerfile
2022-09-20 23:27:36 -07:00
David Dworken
80822c722e
Optimize SQL query so it only runs one query rather than N
2022-09-20 21:36:02 -07:00
David Dworken
19bbf27efd
More reliable handling for deletion requests
2022-09-20 21:28:49 -07:00
David Dworken
a2f95fdc4d
Fix typo in SQL query
2022-09-20 21:15:20 -07:00
David Dworken
41f82e8034
Add ReadCount field to deletion requests so that we can eventually delete them from the DB
2022-09-20 21:13:59 -07:00
David Dworken
5391ecd220
First version of working redaction with passing integration tests
2022-09-19 22:49:48 -07:00
David Dworken
1bf510ff8a
Better offline handling, implemented the local portion of delete, and maybe fixed the bug when running tests on github actions
2022-09-18 09:42:24 -07:00
David Dworken
158c2f2c57
Bump backend version to go1.18
2022-09-03 21:28:47 -07:00
David Dworken
0fac3b7286
Make query params required rather than having weird undefined behavior
2022-06-04 23:03:05 -07:00
David Dworken
b760db58cd
Update URL formats for backend
2022-06-04 20:36:34 -07:00
David Dworken
1da703e9c2
Working update code for macos
2022-05-26 23:45:08 -07:00
David Dworken
47d13a9c27
Passing the basic dump testss
2022-05-22 19:45:46 -07:00
David Dworken
feaa8b2bd1
Add a DB dump test that passes on zsh (is failing for an unknown reason on bash currently) + fix backup and restore for WAL files + better offline support
2022-05-01 22:37:26 -04:00
David Dworken
cbc4e70605
Integrated client-side with dump requests, haven't written any integration tests yet
2022-04-28 11:46:14 -07:00
David Dworken
46d7e9e013
Tests passing when being run offline, still not integrated with the new API endpoints yet
2022-04-28 11:26:55 -07:00
David Dworken
8a018b71b8
Add back in the bootstrap handler since I realized it is still useful to provide some amount of history at first install immediately
2022-04-28 11:05:56 -07:00
David Dworken
0499010981
Remove the 'e' prefix from api endpoints + implement backend APIs for clean loading of all data from other instances
2022-04-28 10:56:59 -07:00
David Dworken
c79a510b9c
Maybe working xattr support for code signing
2022-04-25 22:03:31 -07:00
David Dworken
be27d190c6
Improve the code for the /download API endpoint so that it auto-decrements the tag until it finds a valid version
2022-04-21 22:02:28 -07:00
David Dworken
ec4d64dedf
Delete test that was testing deleted HTTP handlers
2022-04-17 12:56:10 -07:00
David Dworken
f3cbe98091
Fix download URLs for the new git tag format
2022-04-17 12:07:32 -07:00
David Dworken
e98783d30f
Add server-side code to handle updates for non-linux
2022-04-17 12:02:56 -07:00
David Dworken
f22bb5b4d9
Fix TODOs in tests
2022-04-16 23:02:36 -07:00
David Dworken
735a98a611
Add API endpoint for getting information about the latest hishtory version and download URLs
2022-04-16 16:34:09 -07:00
David Dworken
05f1af8714
Add trigger-cron endpoint + optimize dockerfile for server
2022-04-16 16:28:53 -07:00
David Dworken
d7ad96906f
Add todo + remove pending features that are already done
2022-04-16 12:39:28 -07:00
David Dworken
a436edbd16
Add backend code to delete entries that have already been read + start collecitng data on usage data so we can eventually prune data from users that are no longer using hishtory
2022-04-16 11:37:43 -07:00
David Dworken
2b676a935b
Optimize server by only sending new-ish records to the client
2022-04-16 10:30:39 -07:00
David Dworken
c369661e53
More server-side logging + client-side logging
2022-04-16 00:44:47 -07:00
David Dworken
df59b43e03
Add request logs to the backend
2022-04-15 23:19:39 -07:00
David Dworken
ab0d9d4c90
optimize test runtime by running a single server instance per-file rather than one per test
2022-04-15 11:20:23 -07:00
David Dworken
dc6fb6a47b
Refactor tests to make them non-flakey and pass on actions
...
This required adding the ability to have hishtory run synchronously to avoid reconditions. I also added additional waiting code. Also a whole bunch of new tests and disabled gorm's default logger which also caued flakeyness
2022-04-14 20:18:49 -07:00
David Dworken
e2b5fd13bf
tests passing inside act, hopefully on real actions
2022-04-13 21:30:27 -07:00
David Dworken
970e5d75db
add tests for user and hostname atoms + moved server tests to server_test.go
2022-04-10 17:38:20 -07:00
David Dworken
bf789605d1
record registration ip and date in the server-side DB for usage analytics
2022-04-09 16:37:51 -07:00
David Dworken
64935391b6
More tests
2022-04-09 13:02:30 -07:00
David Dworken
b92087fc1d
Try again to fix timezone test
2022-04-09 12:57:58 -07:00
David Dworken
c8ba560e0c
handle github api rate limit, tests for the update command, and fix timezone bug in tests
2022-04-09 12:50:01 -07:00
David Dworken
f7627d3e77
add links to github repo and better format the installation instructions
2022-04-09 00:33:20 -07:00
David Dworken
57a95fb9a7
Moved all backend code into the backend/ directory
2022-04-08 23:56:02 -07:00
