extern/hishtory
1
0
Fork 0
mirror of https://github.com/ddworken/hishtory.git synced 2024-11-25 09:43:20 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,319 Commits 67 Branches 499 Tags 50 MiB
import
Commit Graph

271 Commits

Author SHA1 Message Date
David Dworken
b5db10d56a
Update submission endpoint to only include dump and deletion requests if the request is from a modern client, and update it to increment the relevant counters 2023-09-23 11:34:56 -07:00
David Dworken
47bdc966b2
Add GreaterThan method for comparing versions 2023-09-23 11:34:02 -07:00
David Dworken
6cfbec4823
Add struct + functions for parsing hishtory versions 2023-09-23 11:23:47 -07:00
David Dworken
eb85f9a293
Revert "Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt)" 
This reverts commit bbfda0be96.
 2023-09-22 21:20:53 -07:00
David Dworken
bbfda0be96
Remove TODO: I think this isn't worth implementing since in the long term, clients will update and this branch will not be necessary (and thus will primarily be tech debt) 2023-09-22 19:20:51 -07:00
David Dworken
5bdbd9b262
Revert increased read_count requirement for deletion requests since deleting via encrypted-ID should be reliable 2023-09-22 19:19:06 -07:00
David Dworken
cc11916f3c
Create func to automatically create DB indexes rather than just documenting them in a comment that has to be manually executed 2023-09-22 19:11:43 -07:00
David Dworken
8c6443ed07
Fix incorrect column name 2023-09-22 18:59:37 -07:00
David Dworken
1d878195b2
Rework ff98a7907c to use the newly added EntryId column rather than deleting based on the start time 2023-09-22 14:03:41 -07:00
David Dworken
9b847c5e35
Further optimize client-server roundtrips by including deletion and dump requests in submit responses (follow up to 1e43de689f) 2023-09-22 13:49:29 -07:00
David Dworken
2a5a6d65c4
Roll-forward "Add preliminary support for persisting pre-saved history entries remotely" 
This rolls-forward commit 66916c27cb.
 2023-09-22 13:13:46 -07:00
David Dworken
66916c27cb
Revert "Add preliminary support for persisting pre-saved history entries remotely" 
This reverts commit ff98a7907c. That commit is incomplete since it doesn't include support for the continous deletion of pre-saved history entries as soon as they
finish running. Support for this will require a good bit more work/thought, so reverting for and keeping this code in the git history.
 2023-09-21 12:39:20 -07:00
David Dworken
ff98a7907c
Add preliminary support for persisting pre-saved history entries remotely 2023-09-21 12:39:04 -07:00
David Dworken
a3b865fa6b
Move all handlers into either api_handlers.go for client-exposed handlers, or debug_handlers.go for debug/admin functionality 2023-09-21 11:47:16 -07:00
David Dworken
9fc1140ab9
Swap json decoding to Decoder API for shorter/clearer code 2023-09-21 11:42:02 -07:00
David Dworken
1e43de689f
Optimize number of round-trip HTTP connections made by the client by having the submit handler return metadata about whether there are pending dump/deletion requests 
For now, I'm still keeping the dedicated endpoints for those functionalities, but since most of the time there are no dump/deletion requests this should cut down the number of requests made by the client by 2/3.
 2023-09-21 11:35:24 -07:00
David Dworken
f64362fbba
Fix metrics tag format so handler tags are properly exported to datadog 2023-09-17 22:46:31 -07:00
David Dworken
1ab61f1d7d
Fix missing err != nil branch 2023-09-17 13:07:38 -07:00
David Dworken
d94e1bfa22
Fix imports after moving backend/server code in 772aaf2453 2023-09-17 12:53:55 -07:00
David Dworken
772aaf2453
Move internal/ directory into backend/server since it is all server-specific internals 2023-09-17 12:51:21 -07:00
David Dworken
04a0fc40cd
Make errors from updateUsageData(...) crash when running in non-prod to ensure they're detected/handled before deployment 2023-09-17 12:03:42 -07:00
David Dworken
b733878878
Skip re-initializing tables when running in prod to make server startup faster 2023-09-15 17:51:14 -07:00
David Dworken
63741a0d14
Optimize docker build efficiency by running apk add early so that it is cached 2023-09-15 17:10:55 -07:00
David Dworken
1036e14c17
Add debug prints to debug server startup 2023-09-15 17:07:25 -07:00
David Dworken
a66ea1387d
Refactor server.go to remove two global variables 2023-09-13 21:47:06 -07:00
Sergio Moura
a8360efa67 revert main.ReleaseVersion changes 2023-09-12 15:55:41 -04:00
Sergio Moura
0d30011a33 break down release versions and fix server tests 2023-09-12 15:44:03 -04:00
Sergio Moura
60a0e20dd9 extract server object to its own package 2023-09-12 15:44:03 -04:00
Sergio Moura
02b1e8287d isolate all server handlers into a single struct, without using global variables 2023-09-12 15:44:01 -04:00
David Dworken
50c74e5881
Rename a bunch of DB functions + add error checking for DB table creation (follow up to #104) 2023-09-12 08:30:43 -07:00
Sergio Moura
eaa837d17d fix check for empty array result (II) 2023-09-11 16:37:55 -04:00
Sergio Moura
ff2ac2194d fix %w 2023-09-11 13:03:29 -04:00
Sergio Moura
f9ee8ed189 use DateOnly as local constant instead of using the imported time package 2023-09-11 10:15:42 -04:00
Sergio Moura
3c18f62d99 finishing removing direct DB instructions from http handlers 2023-09-08 10:58:01 -04:00
Sergio Moura
0d6aa081d8 remove direct db instructions from apiRegisterHandler and statsHandler 2023-09-08 10:58:01 -04:00
Sergio Moura
382f234c09 remove usage_data related queries from the main package into a database package 2023-09-08 10:58:01 -04:00
Sergio Moura
a3e6d5a940 docker improvements: 
* use alpine for small docker images
* use docker compose to wait for the database to be ready
* use a single Dockerfile with parameters to determine the correct GOARCH
* update makefile to account for the new GOARCH parameter on the Dockerfile
* remove native-arch-Dockerfile
 2023-09-07 11:06:32 -04:00
David Dworken
aa3c7fc41a
Follow up to #103: assign err from fmt.Fprintf to _ to ensure go-errcheck doesn't flag it 2023-09-07 07:57:20 -07:00
David Dworken
68e3a813c9
Follow up to #103: pull context from r.Context() when used rather than at the start of functions 2023-09-07 07:56:03 -07:00
Sergio Moura
e6d922709d fix tests 2023-09-07 09:50:36 -04:00
Sergio Moura
37670230af Merge remote-tracking branch 'origin/master' into sergio/handlers 2023-09-07 09:33:42 -04:00
Sergio Moura
589b99e500 do not use http.StatusNoContent 2023-09-07 08:34:21 -04:00
David Dworken
2a95553fdf
Fix install.py script for arm7 so installs work on raspberry pis 2023-09-06 21:35:28 -07:00
Sergio Moura
2b1ba7e3ba use single context and always return a status to the client 
api handlers do not need an extra context. http.Request already has a context that is being ignored, so we leverage it and stop creating a new one.
make the endpoints return http.StatusNoContent instead of just closing the connection from the client.
 2023-09-06 11:37:14 -04:00
Sergio Moura
83ad8c7b1f wrap errors with %w instead of using %v 2023-09-05 15:08:55 -04:00
David Dworken
f1aefdbf30
Fix download URL for arm7 2023-02-18 22:29:01 -08:00
David Dworken
27cd544524
Add support for arm7 for #63 to support raspberry pis 2023-02-18 22:00:39 -08:00
David Dworken
8dc5168c64
Set max open conns to 1 for test code to fix race conditions that lead to flaky tests 2023-02-13 20:50:33 -08:00
David Dworken
f502cbee1d
Fix memory leak caused by not closing resp.Body (#62) and also fix the server to work when the DD integration is disabled 2022-12-23 17:17:44 -08:00
David Dworken
06a4bf0bcb
Add code to deep clean the DB to remove entries from people with 1 device that haven't been active in at least 90 days 2022-12-17 21:27:00 -08:00
David Dworken
956377d472
linux arm is called aarch64, see #48 2022-12-12 19:45:54 -08:00
David Dworken
83667a52f4
Fix #51 by changing if --> elif 2022-12-12 19:44:02 -08:00
David Dworken
7c86b812bf
Add support for linux arm64 for #48 2022-12-11 20:39:45 -08:00
David Dworken
857e423e10
Fix linter errors + some general clean up 2022-12-11 20:31:50 -08:00
David Dworken
9ed325e0a5
Add support for limiting the number of registrations to fix #46 2022-12-11 19:42:51 -08:00
David Dworken
35444bf56e
Swap ioutil to non-deprecated alternatives + clean up pre-commit errors 2022-11-27 11:59:06 -08:00
David Dworken
3985010a17
Add test for the healthcheck endpoint 2022-11-26 22:53:14 -08:00
David Dworken
22330f3ee1
Ensure that tests assert a lack of leaked connections 2022-11-26 22:40:43 -08:00
David Dworken
f77eb38618
Rename misleading var 2022-11-26 22:16:11 -08:00
David Dworken
2ba535b650
Call Close() on sql.Rows to prevent connection leak 2022-11-26 22:15:16 -08:00
David Dworken
4e84f49de9
Revert "Move usage data into tx to optimize updating usage data" 
This reverts commit aa501262e6.
 2022-11-26 18:59:32 -08:00
David Dworken
aa501262e6
Move usage data into tx to optimize updating usage data 2022-11-26 18:46:20 -08:00
David Dworken
865ce06b95
Optimize healthcheck handler to avoid unnecessary ping in prod + add prod-only config for connection reuse 2022-11-26 18:33:54 -08:00
David Dworken
a98bff0db8
Optimize query latency by moving the read count incrementing to a background task 2022-11-26 12:10:18 -08:00
David Dworken
c603537137
Optimize the apiSubmit function to ensure that it doesn't lead to errors from large insertions 2022-11-26 10:31:43 -08:00
David Dworken
8d87110405
Optimize the healthcheck analyzer to avoid a full table scan 2022-11-25 20:35:17 -08:00
David Dworken
1ce20157c7
Add datadog integration for monitoring the server 2022-11-25 20:04:40 -08:00
David Dworken
8dd9c1d9e4
Create the feedback table in pg 2022-11-20 00:12:41 -08:00
David Dworken
97d7c7e310
Add last registration date to internal status page 2022-11-19 09:41:40 -08:00
David Dworken
0bd9249df3
Update the healthcheck endpoint so it also writes to the DB to detect future cases of #38 2022-11-17 23:43:25 -08:00
David Dworken
d4efc62df1
Add client-side code to ask for feedback when uninstalling 2022-11-16 21:54:05 -08:00
David Dworken
48c77d88f7
Add endpoint to handle feedback reports 2022-11-16 21:42:07 -08:00
David Dworken
78d33cf437
Make the healthcheck endpoint work for non-global production instance usages to fix #35 2022-11-16 20:58:19 -08:00
David Dworken
fd3d932e9d
Document that the server can use sqlite 2022-11-16 20:54:47 -08:00
David Dworken
4aacd8b5e2
Merge pull request #31 from matthewcmead/sqlite_for_server 
allow for sqlite backend database for low-profile self-hosting deploy…
 2022-11-16 20:50:34 -08:00
David Dworken
35208680d8
Improve debug logs in server.go 2022-11-16 20:26:44 -08:00
m
77af69de86 allow for sqlite backend database for low-profile self-hosting deployments 2022-11-14 11:35:05 -05:00
David Dworken
3e0ed67518
Serve a HTML redirect 2022-11-06 14:59:00 -08:00
David Dworken
9cc485f3b1
Remove the redirect since it breaks the install.py 2022-11-06 14:43:49 -08:00
David Dworken
7ceb303caf
Delete the landing page and instead serve a redirect to the github repo 2022-11-06 14:33:28 -08:00
David Dworken
c2802f40dc
Use wait-for instead of a sleep for the dockerfile for self hosting 2022-11-05 13:46:08 -07:00
David Dworken
9ddae45b03
Add fully tested instructions for self hosting 2022-11-05 13:30:37 -07:00
David Dworken
36f776053b
Add docker-compose file for self hosting to fix #7 2022-11-04 23:41:56 -07:00
David Dworken
846f256fa4
Support a TMPDIR variable in case /tmp/ is mounted noexec 2022-11-03 18:04:16 -07:00
David Dworken
436d432065
Add warning message for the buggy slow version of hishtory 2022-11-02 20:52:37 -07:00
David Dworken
cc2a6046d3
Update demo gif to show search atoms 2022-11-01 22:39:54 -07:00
David Dworken
31ef7a2fdd
Better timing for the demo gif 2022-11-01 22:33:48 -07:00
David Dworken
24be30d1e1
Slow down gif 2022-11-01 19:13:47 -07:00
David Dworken
67bb2e979f
Add demo gif showing the control-r integration 2022-11-01 15:39:46 -07:00
David Dworken
4f708aa58c
Remove hard coded pg password and pull the new rotated password from an environment variable 2022-11-01 12:14:20 -07:00
David Dworken
19fbeda539
Set a correct version number for builds containing slsa-verifier <1.3.1 2022-11-01 11:16:23 -07:00
David Dworken
2efa93762a
Another attempt at getting github actions to pass 2022-11-01 11:03:49 -07:00
David Dworken
d2558c8fb5
Skip slsa updates only for the orphaned versions 2022-11-01 10:51:08 -07:00
David Dworken
2183a63f8f Add an endpoint that is hit to determine if slsa is currently broken 2022-10-31 17:32:27 -07:00
David Dworken
cdc5421a7b Move testutils to a separate package so as to move test-only code out of the main binary 2022-10-27 21:53:47 -07:00
David Dworken
b4936d588c Add a very basic healthcheck handler to the backend 2022-10-23 17:54:07 -07:00
David Dworken
70c02bf0ed Remove basic auth since that is now done by the LB 2022-10-22 11:08:33 -07:00
David Dworken
6b2e9059a4 Optimize the submit endpoint to only use 1 query and 1 insert rather than N 2022-10-18 14:52:35 -07:00
David Dworken
87d3dc4d97 Log the actual remote addr in log lines 2022-10-18 14:35:03 -07:00
David Dworken
ad809efef6 Migrate hishtory from k8s to docker compose 2022-10-18 11:53:35 -07:00
David Dworken
ab95b16ea2 Update comment 2022-10-17 20:38:40 -07:00
David Dworken
0da8021ab7 HTML escape the forced banner to prevent XSS (though this domain has no cookies so there would have been very limited harm from an XSS) 2022-10-15 09:58:47 -07:00
David Dworken
a40e44c985 Add explanation of why expectedPasswordHash is ok 2022-10-15 09:26:54 -07:00
David Dworken
38c185c9f9 Add environ variable to configure the postgres DB 2022-10-15 09:21:10 -07:00
David Dworken
68416f9d0f Better string formatting for the internal analytics endpoint 2022-10-04 12:58:00 -07:00
David Dworken
9d95b4ca70 Add comma separation in stats page 2022-10-02 20:42:10 -07:00
David Dworken
a85315d366 Log hishtory version in internal analytics 2022-10-02 20:39:52 -07:00
David Dworken
b7c64b61c8 Always check gorm interactions for errors 2022-10-02 19:41:00 -07:00
David Dworken
757ebb9547 Prompt people if they run hishtory init and already have a bunch of entries + fix tests + add TODOs + add hishtory version to requests 2022-10-01 09:50:06 -07:00
David Dworken
18ddbf2ca9 Track queries in usage data + better formatting in the internal usage page 2022-09-30 23:38:35 -07:00
David Dworken
98a4f002fa Get RemoteAddr from X-Real-Ip header 2022-09-29 23:51:45 -07:00
David Dworken
f0a3caed1c Add an internal only handler to expose basic usage stats via HTTP and keep a per-device counter of number of history entries for my own vanity stats 2022-09-29 23:32:13 -07:00
David Dworken
a108268925 Add TODOs 2022-09-27 22:15:31 -07:00
David Dworken
5fc3cf3ebc Increase speed 2022-09-24 16:14:05 -07:00
David Dworken
aa4a7310dd Add hishtory.svg from asciinema 2022-09-24 16:12:02 -07:00
David Dworken
b0c4780d48 hishtory --> hiSHtory 2022-09-24 15:30:51 -07:00
David Dworken
63625e94fe Add prominent github link to the website 2022-09-24 01:16:13 -07:00
David Dworken
c18c2502cc Add a complex test with many users 2022-09-22 21:45:20 -07:00
David Dworken
c6d4f1ef68 Fix missing newline in zshrc, add test for hishtory redact prompting, and tag TODOs with plans 2022-09-22 18:22:06 -07:00
David Dworken
ee6680f571 Remove debugging information and give up on that test, and add work around for weird zsh bug with importing 2022-09-22 18:09:51 -07:00
David Dworken
6db8044bec Add server tests for redaction 2022-09-21 21:46:46 -07:00
David Dworken
27157e675d Add TODOs 2022-09-21 19:59:31 -07:00
David Dworken
3e89accf97 Delete the tmp client before downloading to clear macos's signature cache 2022-09-21 19:01:34 -07:00
David Dworken
414d8926f6 Fix incorrect go version in dockerfile 2022-09-20 23:27:36 -07:00
David Dworken
80822c722e Optimize SQL query so it only runs one query rather than N 2022-09-20 21:36:02 -07:00
David Dworken
19bbf27efd More reliable handling for deletion requests 2022-09-20 21:28:49 -07:00
David Dworken
a2f95fdc4d Fix typo in SQL query 2022-09-20 21:15:20 -07:00
David Dworken
41f82e8034 Add ReadCount field to deletion requests so that we can eventually delete them from the DB 2022-09-20 21:13:59 -07:00
David Dworken
5391ecd220 First version of working redaction with passing integration tests 2022-09-19 22:49:48 -07:00
David Dworken
1bf510ff8a Better offline handling, implemented the local portion of delete, and maybe fixed the bug when running tests on github actions 2022-09-18 09:42:24 -07:00
David Dworken
158c2f2c57 Bump backend version to go1.18 2022-09-03 21:28:47 -07:00
David Dworken
d0aaef10ff Update landing pages 2022-06-04 23:29:13 -07:00
David Dworken
e745941350 Stop looping the demo video + increase speed 2022-06-04 23:22:07 -07:00
David Dworken
0fac3b7286 Make query params required rather than having weird undefined behavior 2022-06-04 23:03:05 -07:00
David Dworken
72bd46d4e8 Detect failures to install hishtory in install script 2022-06-04 22:09:30 -07:00
David Dworken
b760db58cd Update URL formats for backend 2022-06-04 20:36:34 -07:00
David Dworken
1da703e9c2 Working update code for macos 2022-05-26 23:45:08 -07:00
David Dworken
47d13a9c27 Passing the basic dump testss 2022-05-22 19:45:46 -07:00
David Dworken
feaa8b2bd1 Add a DB dump test that passes on zsh (is failing for an unknown reason on bash currently) + fix backup and restore for WAL files + better offline support 2022-05-01 22:37:26 -04:00
David Dworken
cbc4e70605 Integrated client-side with dump requests, haven't written any integration tests yet 2022-04-28 11:46:14 -07:00
David Dworken
46d7e9e013 Tests passing when being run offline, still not integrated with the new API endpoints yet 2022-04-28 11:26:55 -07:00
David Dworken
8a018b71b8 Add back in the bootstrap handler since I realized it is still useful to provide some amount of history at first install immediately 2022-04-28 11:05:56 -07:00
David Dworken
0499010981 Remove the 'e' prefix from api endpoints + implement backend APIs for clean loading of all data from other instances 2022-04-28 10:56:59 -07:00
David Dworken
c79a510b9c Maybe working xattr support for code signing 2022-04-25 22:03:31 -07:00
David Dworken
be27d190c6 Improve the code for the /download API endpoint so that it auto-decrements the tag until it finds a valid version 2022-04-21 22:02:28 -07:00
David Dworken
7e4f8e060f Add TODO to install.py, and explicitly set CGO_ENABLED=1 2022-04-19 22:22:04 -07:00
David Dworken
0a03ce3407 Add missing .py extension 2022-04-19 20:44:35 -07:00
David Dworken
6c9236785a Add an easy to use cross-platform install script for that classic curl | bash goodness 2022-04-19 20:36:14 -07:00
David Dworken
ec4d64dedf Delete test that was testing deleted HTTP handlers 2022-04-17 12:56:10 -07:00
David Dworken
f3cbe98091 Fix download URLs for the new git tag format 2022-04-17 12:07:32 -07:00