Commit Graph

56 Commits

Author SHA1 Message Date
kim
815b5291e0
[bugfix] fix inconsistent calculated cache sizes (#2115)
* use calculated exampleTime instead of `time.Now()` to ensure no locale data, retweak cache ratios

* update envparsing test

* update default cache memory to 100MiB

* fix envparsing with latest cache target default

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-08-14 17:08:19 +01:00
kim
91cbcd589e
[performance] remove last of relational queries to instead rely on caches (#2091) 2023-08-10 15:08:41 +01:00
kim
9a291dea84
[performance] add caching of status fave, boost of, in reply to ID lists (#2060) 2023-08-04 12:28:33 +01:00
kim
00adf18c24
[feature] simpler cache size configuration (#2051)
* add automatic cache max size generation based on ratios of a singular fixed memory target

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unused cache max-size config variables

Signed-off-by: kim <grufwub@gmail.com>

* slight ratio tweak

Signed-off-by: kim <grufwub@gmail.com>

* remove unused visibility config var

Signed-off-by: kim <grufwub@gmail.com>

* add secret little ratio config trick

Signed-off-by: kim <grufwub@gmail.com>

* fixed a word

Signed-off-by: kim <grufwub@gmail.com>

* update cache library to remove use of TTL in result caches + slice cache

Signed-off-by: kim <grufwub@gmail.com>

* update other cache usages to use correct interface

Signed-off-by: kim <grufwub@gmail.com>

* update example config to explain the cache memory target

Signed-off-by: kim <grufwub@gmail.com>

* update env parsing test with new config values

Signed-off-by: kim <grufwub@gmail.com>

* do some ratio twiddling

Signed-off-by: kim <grufwub@gmail.com>

* add missing header

* update envparsing with latest defaults

Signed-off-by: kim <grufwub@gmail.com>

* update size calculations to take into account result cache, simple cache and extra map overheads

Signed-off-by: kim <grufwub@gmail.com>

* tweak the ratios some more

Signed-off-by: kim <grufwub@gmail.com>

* more nan rampaging

Signed-off-by: kim <grufwub@gmail.com>

* fix envparsing script

Signed-off-by: kim <grufwub@gmail.com>

* update cache library, add sweep function to keep caches trim

Signed-off-by: kim <grufwub@gmail.com>

* sweep caches once a minute

Signed-off-by: kim <grufwub@gmail.com>

* add a regular job to sweep caches and keep under 80% utilisation

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* add new size library used to libraries section of readme

Signed-off-by: kim <grufwub@gmail.com>

* add better explanations for the mem-ratio numbers

Signed-off-by: kim <grufwub@gmail.com>

* update go-cache

Signed-off-by: kim <grufwub@gmail.com>

* library version bump

Signed-off-by: kim <grufwub@gmail.com>

* update cache.result{} size model estimation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-08-03 11:34:35 +02:00
tobi
2be83fdca5
[feature] Allow users to skip http client tls verification for testing purposes (with appropriately loud warnings) (#2052) 2023-08-01 18:50:17 +01:00
tobi
2796a2e82f
[feature] Hashtag federation (in/out), hashtag client API endpoints (#2032)
* update go-fed

* do the things

* remove unused columns from tags

* update to latest lingo from main

* further tag shenanigans

* serve stub page at tag endpoint

* we did it lads

* tests, oh tests, ohhh tests, oh tests (doo doo doo doo)

* swagger docs

* document hashtag usage + federation

* instanceGet

* don't bother parsing tag href

* rename whereStartsWith -> whereStartsLike

* remove GetOrCreateTag

* dont cache status tag timelineability
2023-07-31 15:47:35 +02:00
kim
ed2477ebea
[performance] cache follow, follow request and block ID lists (#2027) 2023-07-31 11:25:29 +01:00
Vyr Cossont
b874e9251e
[feature] Implement markers API (#1989)
* Implement markers API

Fixes #1856

* Correct import grouping in markers files

* Regenerate Swagger for markers API

* Shorten names for readability

* Cache markers for 6 hours

* Update DB ref

* Update envparsing.sh
2023-07-29 12:49:14 +02:00
Vyr Cossont
22ac4607a1
[feature] Support setting private notes on accounts (#1982)
* Support setting private notes on accounts

* Reformat comment whitespace

* Add missing license headers

* Use apiutil.ParseID

* Rename Note model and cache to AccountNote

* Update golden cache config in test/envparsing.sh

* Rename gtsmodel/note.go to gtsmodel/accountnote.go

* Update AccountNote uniqueness constraint name

Now has same prefix as other indexes on this table.

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2023-07-27 10:30:39 +02:00
Daenney
da8282081c
[feature] Lower remote media cache config duration (#2007)
The old default of 30d can lead to a lot of media getting cached and
significant disk usage, even on small or single person instances. A lot
of deployments decrease this value, to 15 or even less. This is less of
an issue when using object storage, but for local storage which is the
more popular deployment option running out of disk space is unpleasant.

With GoToSocial's aim to fit in small places, this changes the default
to a much more conservative 7 days. In all likelihood people aren't
scrolling that far back in their timeline so this change shouldn't
result in any issue. Existing deployments will only be affected by
this change if the admin hasn't already configured this value, or didn't
bootstrap from the example configuration.
2023-07-21 16:22:59 +02:00
firescry
8d92b2479f
[bugfix] Align default values in the configuration file with the code (#1971)
* Set default value of SMTPFrom to empty string

This parameter should contain proper e-mail address (to be provided by user during configuration).

* Update default values in example/config.yaml

Default values and related comments in example/config.yaml are aligned
with values defined in internal/config/defaults.go.
Small improvements to foramting of config.yaml file.

* Add default value for AdvancedThrottlingRetryAfter to internal/config/defaults.go

AdvancedThrottlingRetryAfter was introduced in 70739d3 (superseriousbusiness/gotosocial#1466).

* Update config.yaml snippets in documentation
2023-07-12 10:28:41 +02:00
tobi
2a99df0588
[feature] enable + document explicit IP dialer allowing/denying (#1950)
* [feature] enable + document explicit IP dialer allowing/denying

* lord have mercy

* allee jonge

* shortcut check ipv6 prefixes

* comment

* separate httpclient_test, export Sanitizer
2023-07-07 16:17:39 +02:00
tobi
e70bf8a6c8
[chore/bugfix] Domain block tidying up, Implement first pass of 207 Multi-Status (#1886)
* [chore/refactor] update domain block processing

* expose domain block import errors a lil better

* move/remove unused query keys
2023-07-07 11:34:12 +02:00
tobi
fc524f8cf1
[feature] Configurable custom css length (#1827)
* [feature] Make accounts custom css length configurable

* test custom css validation
2023-05-25 15:18:15 +02:00
tobi
f5c004d67d
[feature] Add List functionality (#1802)
* start working on lists

* further list work

* test list db functions nicely

* more work on lists

* peepoopeepoo

* poke

* start list timeline func

* we're getting there lads

* couldn't be me working on stuff... could it?

* hook up handlers

* fiddling

* weeee

* woah

* screaming, pissing

* fix streaming being a whiny baby

* lint, small test fix, swagger

* tidying up, testing

* fucked! by the linter

* move timelines to state like a boss

* add timeline start to tests using state

* invalidate lists
2023-05-25 10:37:38 +02:00
Daenney
107237c8e8
[feature] Make client IP logging configurable (#1799) 2023-05-21 16:12:47 +01:00
Dominik Süß
6392e00653
feat: initial tracing support (#1623) 2023-05-09 18:19:48 +01:00
tobi
cbb9e2d3f0
[chore/performance] Make sender multiplier configurable (#1750) 2023-05-08 18:03:38 +01:00
kim
de6e3e5f2a
[performance] refactoring + add fave / follow / request / visibility caching (#1607)
* refactor visibility checking, add caching for visibility

* invalidate visibility cache items on account / status deletes

* fix requester ID passed to visibility cache nil ptr

* de-interface caches, fix home / public timeline caching + visibility

* finish adding code comments for visibility filter

* fix angry goconst linter warnings

* actually finish adding filter visibility code comments for timeline functions

* move home timeline status author check to after visibility

* remove now-unused code

* add more code comments

* add TODO code comment, update printed cache start names

* update printed cache names on stop

* start adding separate follow(request) delete db functions, add specific visibility cache tests

* add relationship type caching

* fix getting local account follows / followed-bys, other small codebase improvements

* simplify invalidation using cache hooks, add more GetAccountBy___() functions

* fix boosting to return 404 if not boostable but no error (to not leak status ID)

* remove dead code

* improved placement of cache invalidation

* update license headers

* add example follow, follow-request config entries

* add example visibility cache configuration to config file

* use specific PutFollowRequest() instead of just Put()

* add tests for all GetAccountBy()

* add GetBlockBy() tests

* update block to check primitive fields

* update and finish adding Get{Account,Block,Follow,FollowRequest}By() tests

* fix copy-pasted code

* update envparsing test

* whitespace

* fix bun struct tag

* add license header to gtscontext

* fix old license header

* improved error creation to not use fmt.Errorf() when not needed

* fix various rebase conflicts, fix account test

* remove commented-out code, fix-up mention caching

* fix mention select bun statement

* ensure mention target account populated, pass in context to customrenderer logging

* remove more uncommented code, fix typeutil test

* add statusfave database model caching

* add status fave cache configuration

* add status fave cache example config

* woops, catch missed error. nice catch linter!

* add back testrig panic on nil db

* update example configuration to match defaults, slight tweak to cache configuration defaults

* update envparsing test with new defaults

* fetch followingget to use the follow target account

* use accounnt.IsLocal() instead of empty domain check

* use constants for the cache visibility type check

* use bun.In() for notification type restriction in db query

* include replies when fetching PublicTimeline() (to account for single-author threads in Visibility{}.StatusPublicTimelineable())

* use bun query building for nested select statements to ensure working with postgres

* update public timeline future status checks to match visibility filter

* same as previous, for home timeline

* update public timeline tests to dynamically check for appropriate statuses

* migrate accounts to allow unique constraint on public_key

* provide minimal account with publicKey

---------

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-03-28 14:03:14 +01:00
tobi
7db81cde44
[feature] Email notifications for new / closed moderation reports (#1628)
* start fiddling about with email sending to allow multiple recipients

* do some fiddling

* notifs working

* notify on closed report

* finishing up

* envparsing

* use strings.ContainsAny
2023-03-19 13:11:46 +01:00
Daenney
5e2bf0bdca
[chore] Improve copyright header handling (#1608)
* [chore] Remove years from all license headers

Years or year ranges aren't required in license headers. Many projects
have removed them in recent years and it avoids a bit of yearly toil.

In many cases our copyright claim was also a bit dodgy since we added
the 2021-2023 header to files created after 2021 but you can't claim
copyright into the past that way.

* [chore] Add license header check

This ensures a license header is always added to any new file. This
avoids maintainers/reviewers needing to remember to check for and ask
for it in case a contribution doesn't include it.

* [chore] Add missing license headers

* [chore] Further updates to license header

* Use the more common // indentend comment format
* Remove the hack we had for the linter now that we use the // format
* Add SPDX license identifier
2023-03-12 16:00:57 +01:00
Daenney
e397272fe8
[feature] Discover webfinger through host-meta (#1588)
* [feature] Discover webfinger through host-meta

This implements a fallback for discovering the webfinger endpoint in
case the /.well-known/webfinger endpoint wasn't properly redirected.
Some instances do this because the recommendation used to be to use
host-meta for the webfinger redirect in the before times.

Closes #1558.

* [bug] Ensure we only ever update cache on success

* [chore] Move finger tests to their own place

This adds a test suite for transport and moves the finger cache tests
into there instead of abusing the search test suite.

* [chore] cleanup the test a bit more

We don't really need a separate function for the oddly located webfinger
response as we check the full URL string anyway

* Address review comments

* [chore] update config example

* [chore] access DB only through state in controller
2023-03-08 12:57:41 +00:00
Daenney
d2f6de0185
[feature] Allow loading TLS certs from disk (#1586)
Currently, GtS only supports using the built-in LE client directly for
TLS. However, admins may still want to use GtS directly (so without a
reverse proxy) but with certificates provided through some other
mechanism. They may have some centralised way of provisioning these
things themselves, or simply prefer to use LE but with a different
challenge like DNS-01 which is not supported by autocert.

This adds support for loading a public/private keypair from disk instead
of using LE and reconfigures the server to use a TLS listener if we
succeed in doing so.

Additionally, being able to load TLS keypair from disk opens up the path
to using a custom CA for testing purposes avoinding the need for a
constellation of containers and something like Pebble or Step CA to
provide LE APIs.
2023-03-04 17:24:02 +00:00
kim
a8e6bdfa33
[performance] cache media attachments (#1525)
* replace concurrency worker pools with base models in State.Workers, update code and tests accordingly

* add media attachment caching, slightly tweak default cache config

* further tweak default cache config values

* replace other media attachment db calls to go through cache

* update envparsing test

* fix delete media attachment sql

* fix media sql query

* invalidate cached media entries during status create / update

* fix envparsing test

* fix typo in panic log message...

* add 'updated_at' column during UpdateAttachment

* remove unused func

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-03-03 23:02:23 +00:00
Daenney
68e6d08c76
[feature] Add a request ID and include it in logs (#1476)
This adds a lightweight form of tracing to GTS. Each incoming request is
assigned a Request ID which we then pass on and log in all our log
lines. Any function that gets called downstream from an HTTP handler
should now emit a requestID=value pair whenever it logs something.

Co-authored-by: kim <grufwub@gmail.com>
2023-02-17 12:02:29 +01:00
tobi
40bc03e717
[chore/performance] Update media prune logic, add extra CLI command (#1474)
* start updating media prune stuff a wee bit

* continue prune / uncache work

* more tidying + consistency stuff

* add prune CLI command

* docs

* arg
2023-02-11 12:48:38 +01:00
tobi
782169da76
[chore] set max open / idle conns + conn max lifetime for both postgres and sqlite (#1369)
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite

* reduce cache size default to 8MiB, reduce connections to 2 * cpu

* introduce max open conns multiplier, tune sqlite and pg separately

* go fmt
2023-01-26 15:12:48 +01:00
f0x52
17eecfb6d9
[feature] Public list of suspended domains (#1362)
* basic rendered domain blocklist (unauthenticated!)

* style basic domain block list

* better formatting for domain blocklist

* add opt-in config option for showing suspended domains

* format/linter

* re-use InstancePeersGet for web-accessible domain blocklist

* reword explanation, border styling

* always attach blocklist handler, update error message

* domain blocklist error message grammar
2023-01-25 18:06:41 +01:00
tobi
974ec80a20
[chore] Change default sqlite busy timeout to 5m (#1352) 2023-01-18 14:37:52 +01:00
tobi
627b8eeae6
[feature] Tune sqlite pragmas (#1349)
* sqlite pragma tuning

* use formatuint

* add sqlite busy timeout

* fix incorrect cache size format

* update envparsing test

* add sqlite tuning flags to cli

* set sqlite timeout to 30s default
2023-01-17 12:29:44 +00:00
tobi
d6487933c7
[feature] Implement Report database model and utility functions (#1310)
* implement report database model

* implement report cache + config changes

* implement report database functions

* report uri / regex functions

* update envparsing test

* remove unnecessary uri index

* remove unused function + cache lookup

* process error when storing report
2023-01-10 14:19:05 +00:00
tobi
0dbe6c514f
[chore] Update/add license headers for 2023 (#1304) 2023-01-05 12:43:00 +01:00
tobi
90a14abb0c
[feature] HTTP request throttling middleware (#1297)
* [feature] Add throttling middleware to AP endpoints

* refactor a lil bit

* use config setting, start updating docs

* doc updates

* use relative links in faq doc

* small docs fixes

* return code 503 instead of 429 when throttled

* throttle other endpoints too

* simplify token channel prefills
2023-01-04 11:57:59 +01:00
tobi
941893a774
[chore] The Big Middleware and API Refactor (tm) (#1250)
* interim commit: start refactoring middlewares into package under router

* another interim commit, this is becoming a big job

* another fucking massive interim commit

* refactor bookmarks to new style

* ambassador, wiz zeze commits you are spoiling uz

* she compiles, we're getting there

* we're just normal men; we're just innocent men

* apiutil

* whoopsie

* i'm glad noone reads commit msgs haha :blob_sweat:

* use that weirdo go-bytesize library for maxMultipartMemory

* fix media module paths
2023-01-02 12:10:50 +00:00
kim
cb2b2fd805
[feature] support configuring database caches (#1246)
* update config generator to support nested structs, add cache configuration options

* update envparsing test

* add cache configuration to config parse tests

* set cache configuration in testrig

* move caches to sub-cache "gts" namespace, update envparsing, add cache config docs to example config

Signed-off-by: kim <grufwub@gmail.com>
2022-12-11 13:03:15 +00:00
Dominik Süß
199b685f43
[feature] overhaul the oidc system (#961)
* [feature] overhaul the oidc system

this allows for more flexible username handling and prevents account
takeover using old email addresses

* [feature] add migration path for old OIDC users

* [feature] nicer error reporting for users

* [docs] document the new OIDC flow

* [fix] return early on oidc error

* [docs]: add comments on the finalization logic
2022-12-06 14:15:56 +01:00
Mara Sophie Grosch
948e90b95a
[feature] S3: add config flag to proxy S3 media (#1014)
* S3: add config value "proxy" for not redirecting

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>

* S3: document new config value "proxy"

* S3: add new config value "proxy" to test scripts

Signed-off-by: Mara Sophie Grosch <littlefox@lf-net.org>
2022-11-11 12:03:18 +01:00
tobi
4d66fb9603
[feature] Make rate limit requests amount configurable (#966)
* update rate limit documentation

* regenerate landingpage config helpers

* make rate limit rate configurable
2022-11-06 09:47:48 +00:00
Mia Heidenstedt
f3fc040c2e
[feature] Allow user to show instead of landing page on / (#922)
* configurable user instead of landing page

* rename DefaultUser to LandingPageUser

* code review changes and fix tests

* try to fix pipeline error

* code review changes

* code review changes

* code review changes

* remove unnecessary line
2022-11-03 13:15:31 +01:00
tobi
f0bf69d4d0
[chore] Add ipv6 localhost to trusted proxies by default (#868)
* add ipv6 localhost to trusted proxies by default

* test different trusted proxies
2022-09-29 17:08:56 +02:00
tobi
69a193dae5
[feature] Allow delivery to sharedInboxes where possible (#847)
* update Activity

* add instance-deliver-to-shared-inboxes setting

* update activity version again

* add SharedInboxURI field to accounts

* serdes for endpoints/sharedInbox

* deliver to sharedInbox if one is available

* update tests

* only assign shared inbox if shared domain

* look for shared inbox if currently nil

* go fmt

* finger to get params.RemoteAccountID if necessary

* make comments clearer

* compare dns more consistently
2022-09-23 20:27:35 +01:00
tobi
b42469e4e0
[feature] Allow users to set custom css for their profiles + threads (#808)
* add custom css account property + db func to fetch

* allow account to get/set custom css

* serve custom css for an account

* go fmt

* use monospace for customcss, add link

* add custom css to account cache

* fix broken field

* add custom css docs to user guide

* add `accounts-allow-custom-css` config flag

* add allow custom css to /api/v1/instance response

* only show/set custom css if allowed to do so

* only set/serve custom account css if enabled

* update swagger docs

* chain promise

* make bool a bit clearer

* use cache for GetAccountCustomCSSByUsername
2022-09-12 13:14:29 +02:00
tobi
268f252e0d
[feature] Fetch + display custom emoji in statuses from remote instances (#807)
* start implementing remote emoji fetcher

* update status where pk

* aaa

* tidy up a little

* check size limits for emojis

* thank you linter, i love you <3

* update swagger docs

* add emoji dereference test

* make emoji max sizes configurable

* normalize db.ErrAlreadyExists
2022-09-12 13:03:23 +02:00
tobi
40f91d052c
[chore] Update image/video size defaults to mastodon's (#723) 2022-07-20 12:25:06 +02:00
Dominik Süß
9d0df426da
[feature] S3 support (#674)
* feat: vendor minio client

* feat: introduce storage package with s3 support

* feat: serve s3 files directly

this saves a lot of bandwith as the files are fetched from the object
store directly

* fix: use explicit local storage in tests

* feat: integrate s3 storage with the main server

* fix: add s3 config to cli tests

* docs: explicitly set values in example config

also adds license header to the storage package

* fix: use better http status code on s3 redirect

HTTP 302 Found is the best fit, as it signifies that the resource
requested was found but not under its presumed URL

307/TemporaryRedirect would mean that this resource is usually located
here, not in this case

303/SeeOther indicates that the redirection does not link to the
requested resource but to another page

* refactor: use context in storage driver interface
2022-07-03 12:08:30 +02:00
tobi
5f00d4980b
[feature] Implement /api/v1/instance/peers endpoint (#660)
* add missing license headers

* start adding instance peers get

* rename domainblock.go

* embed domain in domainblock so it can be reused

* update swagger docs

* add test instances to db

* update tests

* add/update instancepeersget

* update domain model

* add getinstancepeers to db

* instance-expose-peers, instance-expose-suspended

* add auth checks for both current filters

* attach endpoint to router

* include public comment

* obfuscate domain if required

* go mod tidy

* update swagger docs

* remove unnecessary comment

* return 'flat' peerlist if no query params provided
2022-06-23 16:54:54 +02:00
tobi
327d3f001f
[feature] Start adding advanced configuration options, starting with samesite (#628)
* fix incorrect port being used for db

* start adding advanced config flags

* use samesite lax by default
2022-06-03 15:40:38 +02:00
kim
43ac0cdb9c
[chore] Global server configuration overhaul (#575)
* move config flag names and usage to config package, rewrite config package to use global Configuration{} struct

Signed-off-by: kim <grufwub@gmail.com>

* improved code comment

Signed-off-by: kim <grufwub@gmail.com>

* linter

Signed-off-by: kim <grufwub@gmail.com>

* fix unmarshaling

Signed-off-by: kim <grufwub@gmail.com>

* remove kim's custom go compiler changes

Signed-off-by: kim <grufwub@gmail.com>

* generate setter and flag-name functions, implement these in codebase

Signed-off-by: kim <grufwub@gmail.com>

* update deps

Signed-off-by: kim <grufwub@gmail.com>

* small change

Signed-off-by: kim <grufwub@gmail.com>

* appease the linter...

Signed-off-by: kim <grufwub@gmail.com>

* move configuration into ConfigState structure, ensure reloading to/from viper settings to keep in sync

Signed-off-by: kim <grufwub@gmail.com>

* lint

Signed-off-by: kim <grufwub@gmail.com>

* update code comments

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* fix merge issue

Signed-off-by: kim <grufwub@gmail.com>

* improved version string (removes time + go version)

Signed-off-by: kim <grufwub@gmail.com>

* fix version string build to pass test script + consolidate logic in func

Signed-off-by: kim <grufwub@gmail.com>

* add license text, update config.Defaults comment

Signed-off-by: kim <grufwub@gmail.com>

* add license text to generated config helpers file

Signed-off-by: kim <grufwub@gmail.com>

* defer unlock on config.Set___(), to ensure unlocked on panic

Signed-off-by: kim <grufwub@gmail.com>

* make it more obvious which cmd flags are being attached

Signed-off-by: kim <grufwub@gmail.com>
2022-05-30 14:41:24 +02:00
tobi
094f032f74
[feature] Add log-db-queries config option (#465) 2022-04-18 16:47:11 +02:00
tobi
07727753b9
[feature] Clean up/uncache remote media (#407)
* Add whereNotEmptyAndNotNull

* Add GetRemoteOlderThanDays

* Add GetRemoteOlderThanDays

* Add PruneRemote to Manager interface

* Start implementing PruneRemote

* add new attachment + status to tests

* fix up and test GetRemoteOlderThan

* fix bad import

* PruneRemote: return number pruned

* add Cached column to mediaattachment

* update + test pruneRemote

* update mediaTest

* use Cached column

* upstep bun to latest version

* embed structs in mediaAttachment

* migrate mediaAttachment to new format

* don't default cached to true

* select only remote media

* update db dependencies

* step bun back to last working version

* update pruneRemote to use Cached field

* fix storage path of test attachments

* add recache logic to manager

* fix trimmed aspect ratio

* test prune and recache

* return errwithcode

* tidy up different paths for emoji vs attachment

* fix incorrect thumbnail type being stored

* expose TransportController to media processor

* implement tee-ing recached content

* add thoughts of dog to test fedi attachments

* test get remote files

* add comment on PruneRemote

* add postData cleanup to recache

* test thumbnail fetching

* add incredible diagram

* go mod tidy

* buffer pipes for recache streaming

* test for client stops reading after 1kb

* add media-remote-cache-days to config

* add cron package

* wrap logrus so it's available to cron

* start and stop cron jobs gracefully
2022-03-07 11:08:26 +01:00