Commit Graph

219 Commits

Author SHA1 Message Date
kim
21bb324156
[chore] media and emoji refactoring (#3000)
* start updating media manager interface ready for storing attachments / emoji right away

* store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load()

* remove now unused media workers

* fix tests and issues

* fix another test!

* fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues

* fix more tests

* fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis

* whoops, rebase issue

* remove kim's whacky experiments

* do some reshuffling, ensure emoji uri gets set

* ensure marked as not cached on cleanup

* tweaks to media / emoji processing to handle context canceled better

* ensure newly fetched emojis actually get set in returned slice

* use different varnames to be a bit more obvious

* move emoji refresh rate limiting to dereferencer

* add exported dereferencer functions for remote media, use these for recaching in processor

* add check for nil attachment in updateAttachment()

* remove unused emoji and media fields + columns

* see previous commit

* fix old migrations expecting image_updated_at to exists (from copies of old models)

* remove freshness checking code (seems to be broken...)

* fix error arg causing nil ptr exception

* finish documentating functions with comments, slight tweaks to media / emoji deref error logic

* remove some extra unneeded boolean checking

* finish writing documentation (code comments) for exported media manager methods

* undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot

* move doesColumnExist() to util.go in migrations package
2024-06-26 16:01:16 +01:00
dependabot[bot]
03c5d016a7
[chore]: Bump github.com/gorilla/feeds from 1.1.2 to 1.2.0 (#3035) 2024-06-24 09:04:27 +00:00
tobi
611f9de39b
[bugfix] Deref stats async, serve stub collections if handshaking (#2990)
* [bugfix] Deref stats async, allow peek if handshaking

* don't return totalItems when handshaking or hiding collections

* use GetLimit()

* use StubAccountStats
2024-06-11 11:54:59 +02:00
kim
fd6637df4a
[bugfix] boost and account recursion (#2982)
* fix possible infinite recursion if moved accounts are self-referential

* adds a defensive check for a boost being a boost of a boost wrapper

* add checks on input for a boost of a boost

* remove unnecessary check

* add protections on account move to prevent move recursion loops

* separate status conversion without boost logic into separate function to remove risk of recursion

* move boost check to boost function itself

* formatting

* use error 422 instead of 500

* use gtserror not standard errors package for error creation
2024-06-10 19:42:41 +01:00
tobi
ce3b8aacf7
[chore] Warn about email/password change when using OIDC (#2975)
* [chore] Warn about email/password change when using OIDC

* go fmt
2024-06-07 15:21:57 +01:00
Vyr Cossont
e605788b4b
[bugfix] Filter fixes (#2971)
* Add Filter.Expired convenience method, consistent with mutes

* Add missing Swagger for filter keyword/status delete and for 403s on moved accounts
2024-06-06 20:16:20 +02:00
Vyr Cossont
5e2d4fdb19
[feature] User muting (#2960)
* User muting

* Address review feedback

* Rename uniqueness constraint on user_mutes to match convention

* Remove unused account_id from where clause

* Add UserMute to NewTestDB

* Update test/envparsing.sh with new and fixed cache stuff

* Address tobi's review comments

* Make compiledUserMuteListEntry.expired consistent with UserMute.Expired

* Make sure mute_expires_at is serialized as an explicit null for indefinite mutes

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-06-06 16:38:02 +00:00
tobi
bcda048eab
[feature] Self-serve email change for users (#2957)
* [feature] Email change

* frontend stuff for changing email

* docs

* tests etc

* differentiate more clearly between local user+account and account

* populate user
2024-06-06 14:43:25 +01:00
kim
5dcc954072
[feature] do not uncache status / emoji media if attached status is bookmarked (#2956)
* do not uncache status / emoji media if attached status is bookmarked

* add status bookmark and bookmark IDs caches

* update status bookmark tests

* move IsStatusBookmarkedBy() to StatusBookmark{} interface, rely on cache

* fix envparsing.sh test
2024-06-06 11:44:43 +01:00
tobi
6f26b32ec3
[bugfix] Don't nil emojis + fields on blocked accounts (#2968)
* [bugfix] Don't nil emojis + fields on blocked accounts

* comment

* swagger
2024-06-06 11:22:16 +01:00
tobi
d79c2f2688
[feature/frontend] Add debug sections to settings panel (#2950)
* [feature/frontend] Add debug sections to settings panel

* max-width

* swagger
2024-06-03 11:20:53 +02:00
Vyr Cossont
61a8d36255
[feature] Implement Filter API v2 (#2936)
* Use correct entity name

* We support server-side filters now

* Document filter v1 methods that can throw a 409

* Validate v1 filter phrase as filter title

* Always check v1 filter API status codes in tests

* Document keyword minimum requirement on filter API v1

* Make it possible to specify filter keyword update columns per filter keyword

* Implement v2 filter API

* Fix lint and tests

* Update Swagger spec

* Fix filter update test

* Update Swagger spec *correctly*

* Update actual files Swagger spec was generated from

* Remove keywords_attributes and statuses_attributes

* Add test for serialization of empty filter

* More helpful messages when object is owned by wrong account
2024-05-31 12:55:56 +02:00
Vyr Cossont
975e92b7f1
[feature] Implement profile API (#2926)
* Implement profile API

This Mastodon 4.2 extension provides capabilities missing from the existing Mastodon account update API: deleting an account's avatar or header.

See: https://docs.joinmastodon.org/methods/profile/

* Move profile media methods to media processor

* Remove check for moved account
2024-05-29 12:57:44 +02:00
kim
b092da6d28
[performance] cache v2 filter keyword regular expressions (#2903)
* add caching of filterkeyword regular expressions

* formatting

* fix WholeWord nil check
2024-05-21 14:20:19 +01:00
Vyr Cossont
45f4afe60e
feature: filters v2 server-side warning/hiding (#2793)
* Remove dead code

* Filter statuses when converting to frontend representation

* status.filtered is an array

* Make matching case-insensitive

* Remove TODOs that don't need to be done now

* Add missing filter check for notification

* lint: rename ErrHideStatus

* APIFilterActionToFilterAction not used yet

* swaggerino docseroni

* Address review comments

* Add apimodel.FilterActionNone

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-05-06 12:49:08 +01:00
tobi
b7c629a18a
[bugfix] Fix incorrect field name for status source, add helpful message (#2854)
* [bugfix] Fix incorrect field name for status source, add helpful message

* swagger

* yyammm
2024-04-18 13:22:55 +02:00
tobi
cef9924d9a
[feature] Status source endpoint (#2848)
* [feature] statusSource endpoint

* finish up
2024-04-17 13:49:20 +01:00
tobi
8cf685fbe9
[bugfix] Fix minor API issue w/ boosted statuses (#2846) 2024-04-17 12:41:40 +02:00
tobi
3cceed11b2
[feature/performance] Store account stats in separate table (#2831)
* [feature/performance] Store account stats in separate table, get stats from remote

* test account stats

* add some missing increment / decrement calls

* change stats function signatures

* rejig logging a bit

* use lock when updating stats
2024-04-16 13:10:13 +02:00
tobi
9fb8a78f91
[feature] New user sign-up via web page (#2796)
* [feature] User sign-up form and admin notifs

* add chosen + filtered languages to migration

* remove stray comment

* chosen languages schmosen schmanguages

* proper error on local account missing
2024-04-11 11:45:53 +02:00
tobi
4bbdef02f1
[chore] Try to parse public key as both Actor + bare key (#2710)
* [chore] Try to parse public key as both Actor + bare key

* fix weird test fail
2024-04-02 11:30:10 +01:00
tobi
f05874be30
[feature] Option to hide followers/following (#2788) 2024-04-02 10:42:24 +01:00
tobi
8953f57d88
[feature] User-selectable preset CSS themes for accounts (#2777)
* [feature] User-selectable preset themes

* docs, more theme stuff

* lint, tests

* fix css name

* correct some little issues

* add another theme

* fix poll background

* okay last theme i swear

* make retrieval of apimodel themes more conventional

* preallocate stylesheet slices
2024-03-25 17:32:24 +00:00
tobi
7f4a0a1aeb
[chore] Move local account settings to separate db table (#2770)
* [chore] Move local account settings to separate database model

* don't use separate settings_id
2024-03-22 14:03:46 +01:00
Vyr Cossont
61a2b91f45
[feature] Filters v1 (#2594)
* Implement client-side v1 filters

* Exclude linter false positives

* Update test/envparsing.sh

* Fix minor Swagger, style, and Bun usage issues

* Regenerate Swagger

* De-generify filter keywords

* Remove updating filter statuses

This is an operation that the Mastodon v2 filter API doesn't actually have, because filter statuses, unlike keywords, don't have options: the only info they contain is the status ID to be filtered.

* Add a test for filter statuses specifically

* De-generify filter statuses

* Inline FilterEntry

* Use vertical style for Bun operations consistently

* Add comment on Filter DB interface

* Remove GoLand linter control comments

Our existing linters should catch these, or they don't matter very much

* Reduce memory ratio for filters
2024-03-06 11:15:58 +01:00
tobi
9cadc764b3
[feature] Add experimental instance-federation-spam-filter option (#2685)
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
kim
1d51e3c8d6
[bugfix] 2643 bug search for account url doesnt always work when redirected (#2673)
* update activity library so dereferencer returns full response and checks *final* link to allow for redirects

* temporarily add bodged fixed library

* remove unused code

* update getAccountFeatured() to use dereferenceCollectionPage()

* make sure to release map

* perform a 2nd decode to ensure reader is empty after primary decode

* add comment explaining choice of using Decode() instead of Unmarshal()

* update embedded activity library to latest matching https://github.com/superseriousbusiness/activity/pull/21

* add checks to look for changed URI and re-check database if redirected

* update max iteration count to 512, add checks during dereferenceAncestors() for indirect URLs

* remove doubled-up code

* fix use of status instead of current

* use URIs for checking equality for security

* use the latest known URI for boost_of_uri in case original was an indirect

* add dereferenceCollection() function for dereferenceAccountFeatured()

* pull in latest github.com/superseriousbusiness/activity version (and remove the bodge!!)

* fix typo in code comments

* update decodeType() to accept a readcloser and handle body closing

* switch to checking using BoostOfID and add note why not using BoostOfURI

* ensure InReplyTo gets unset when deleting status parent in case currently stubbed

* add tests for Collection and CollectionPage iterators
2024-02-23 16:24:40 +01:00
tobi
8cafa6b74b
[feature] Add requested_by to relationship model (#2672)
* [feature] Add `requested_by` to relationship model

* whoops, missed some tests
2024-02-20 17:50:54 +00:00
tobi
d10226e912
[bugfix] Use ptr for instance stats entries to avoid skipping 0 values (#2666)
* [bugfix] Use ptr for instance stats entries to avoid skipping 0 values

* comment explaining why stats values are pointers
2024-02-19 13:17:14 +01:00
tobi
aa396c78d3
[feature] serdes for moved/also_known_as (#2600)
* [feature] serdes for moved/also_known_as

* document `alsoKnownAs` and `movedTo` properties

* only implicitly populate AKA uris from DB for local accounts

* don't let remotes store more than 20 AKA uris to avoid shenanigans
2024-02-06 09:45:46 +00:00
tobi
aa8bbe6ad2
[bugfix] Fix Postgres emoji delete, emoji category change (#2570)
* [bugfix] Fix Postgres emoji delete, emoji category change

* revert trace logging

* caching issue

* update tests
2024-01-29 14:57:22 +00:00
tobi
e3052e8c82
[bugfix] Don't return Account or Status if new and dereferencing failed, other small fixes (#2563)
* tidy up account, status, webfingering logic a wee bit

* go fmt

* invert published check

* alter resp initialization

* get Published from account in typeutils

* don't instantiate error for no darn good reason

* shadow err

* don't repeat error codes in wrapped errors

* don't wrap error unnecessarily
2024-01-26 14:17:10 +01:00
tobi
b2cacd6b01
[bugfix] Prevent URL + URI for same account being used as alias target (#2545)
* [bugfix] Ensure URL and URI for same account can't both be provided as alias

* test whoopsie from previous PR
2024-01-20 12:45:43 +01:00
tobi
5ca86b1c57
[chore] Harden up boolptr logic on Accounts, warn if not set (#2544) 2024-01-19 13:02:04 +00:00
kim
7ec1e1332e
[performance] overhaul struct (+ result) caching library for simplicity, performance and multiple-result lookups (#2535)
* rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial

* use actual go-structr release version (not just commit hash)

* revert go toolchain changes (damn you go for auto changing this)

* fix go mod woes

* ensure %w is used in calls to errs.Appendf()

* fix error checking

* fix possible panic

* remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop

* fix copy-paste artifact... 😇

* fix all comment copy-paste artifacts

* remove dropID() function, now we can just use slices.DeleteFunc()

* use util.Deduplicate() instead of collate(), move collate to util

* move orderByIDs() to util package and "generify"

* add a util.DeleteIf() function, use this to delete entries on failed population

* use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol)

* add note about how collate differs from deduplicate
2024-01-19 12:57:29 +00:00
tobi
c36f9ac37b
[feature] Account alias / move API + db models (#2518)
* [feature] Account alias / move API + db models

* go fmt

* fix little cherry-pick issues

* update error checking, formatting

* add and use new util functions to simplify alias logic
2024-01-16 16:22:44 +00:00
tobi
7cce1a7cc6
[bugfix] Outdent placeholder from <aside> when returning unknown media attachments (#2485)
* [bugfix] Outdent placeholder from `<aside>`

* visually separate placeholder note from body

* fix test
2024-01-09 10:41:32 +01:00
tobi
d5e3996a18
[feature] Parse instance descriptors as markdown, show T&C on /about (#2481)
* [feature] Parse instance descriptors as markdown, show T&C on /about

* lint

* remove unnecessary nullzero tags
2024-01-05 13:39:31 +01:00
tobi
0ff52b71f2
[chore] Refactor HTML templates and CSS (#2480)
* [chore] Refactor HTML templates and CSS

* eslint

* ignore "Local"

* rss tests

* fiddle with OG just a tiny bit

* dick around with polls a bit more so SR stops saying "clickable"

* remove break

* oh lord

* don't lazy load avatar

* fix ogmeta tests

* clean up some cruft

* catch remaining calls to c.HTML

* fix error rendering + stack overflow in tag

* allow templating attributes

* fix indent

* set aria-hidden on status complementary content, since it's already present in the label anyway

* tidy up templating calls a little

* try to make styling a bit more consistent + readable

* fix up some remaining CSS issues

* fix up reports
2023-12-27 11:23:52 +01:00
Sam Lade
07bd848028
fix poll total vote double count (#2464) 2023-12-16 19:12:25 +00:00
kim
ac48192562
[bugfix] poll vote count fixes (#2444)
* don't drop all vote counts if hideCounts is set, refactors poll option extraction slightly

* omit voters_count when not set

* make voters_count a ptr to ensure it is omit unless definitely needed

* handle case of expires_at, voters_count and option.votes_count being nilable

* faster isNil check

* remove omitempty tags since mastodon API marks things as nullable but still sets them in outgoing json
2023-12-12 13:47:07 +00:00
tobi
d60edf7ec6
[bugfix] Ensure pre renders as expected, fix orderedCollectionPage (#2434) 2023-12-10 12:36:00 +01:00
tobi
cc91ea057d
[bugfix] Fix web media not showing as sensitive (#2433)
* [bugfix] Fix web media not showing as sensitive

* test

* go fmt
2023-12-09 16:54:38 +01:00
tobi
dacfd413dc
[chore/frontend] Refactor status templates slightly, put polls behind CWs if present (#2419)
* [chore/frontend] Reorder templates, allow polls to be inside content warnings

* show when multiple-choice
2023-12-05 12:43:07 +01:00
tobi
0e2c342191
[bugfix/chore] Announce reliability updates (#2405)
* [bugfix/chore] `Announce` updates

* test update

* fix tests

* TestParseAnnounce

* update comments

* don't lock/unlock, change function signature

* naming stuff

* don't check domain block twice

* UnwrapIfBoost

* beep boop
2023-12-01 15:27:15 +01:00
kim
eb170003b8
[bugfix] return 400 Bad Request on more cases of malformed AS data (#2399) 2023-11-30 16:22:34 +00:00
tobi
e4e0a5e3f6
[bugfix] Add Actor to outgoing poll vote Create; other fixes (#2384) 2023-11-27 13:14:28 +00:00
tobi
71e0aaee3c
[bugfix/chore] id poll options properly (#2379)
* aria-labels for polls, id poll options properly

* my bad, aria labels are for interactive elements
2023-11-22 16:27:32 +01:00
tobi
8c2d94c168
[feature] Poll web view (#2377)
* [feature] Render polls nicely on the web view

* use figure for poll, other small tweaks

* reverse share + count (lines up better)

* poll options list entries

* fix up some remaining things
2023-11-22 12:17:42 +01:00
tobi
cfefbc08d8
[feature] Federate status language in and out (#2366)
* [feature] Federate status language in + out

* go fmt

* tests, little fix

* improve comments

* unnest a bit

* avoid unnecessary nil check

* use more descriptive variable for contentMap

* prefer instance languages when selecting from contentMap

* update docs to reflect lang selection

* rename rdfLangString -> rdfLangs

* update comments to mention Pollable

* iter through slice instead of map
2023-11-21 15:13:30 +01:00