extern/gotosocial
1
1
Fork 0
mirror of https://github.com/superseriousbusiness/gotosocial.git synced 2025-01-05 05:39:00 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into push-notifications

Browse Source
This commit is contained in:
Vyr Cossont 2024-12-25 17:42:56 -08:00
commit 25dc70e0df
256 changed files with 7891 additions and 3439 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/ISSUE_TEMPLATE/bug_frontend.yaml vendored
View File

@ -1,6 +1,6 @@
name: Frontend Bug Report
description: Report an issue related to the web frontend
title: "[bug] Issue Title"
title: "[bug/frontend] Issue Title"
labels: ["bug", "frontend"]
assignees: []

10
CONTRIBUTING.md
View File

@ -24,7 +24,7 @@ These contribution guidelines were adapted from / inspired by those of Gitea (ht
- [Finding your way around the code](#finding-your-way-around-the-code)
- [Style / Linting / Formatting](#style--linting--formatting)
- [Testing](#testing)
- [Standalone Testrig with Semaphore](#standalone-testrig-with-semaphore)
- [Standalone Testrig with Pinafore](#standalone-testrig-with-pinafore)
- [Running automated tests](#running-automated-tests)
- [SQLite](#sqlite)
- [Postgres](#postgres)
@ -401,9 +401,9 @@ GoToSocial provides a [testrig](https://github.com/superseriousbusiness/gotosoci
One thing that *isn't* mocked is the Database interface because it's just easier to use an in-memory SQLite database than to mock everything out.
#### Standalone Testrig with Semaphore
#### Standalone Testrig with Pinafore
You can launch a testrig as a standalone server running at localhost, which you can connect to using something like [Semaphore](https://github.com/NickColley/semaphore/).
You can launch a testrig as a standalone server running at localhost, which you can connect to using something like [Pinafore](https://github.com/nolanlawson/pinafore/).
To do this, first build the gotosocial binary with `DEBUG=1 ./scripts/build.sh`.
@ -413,14 +413,14 @@ Then, launch the testrig with the `DEBUG` environment variable set by invoking t
DEBUG=1 ./gotosocial testrig start
```
To run Semaphore locally in dev mode, first clone the [Semaphore](https://github.com/NickColley/semaphore/) repository, and then run the following commands in the cloned directory:
To run Pinafore locally in dev mode, first clone the [Pinafore](https://github.com/nolanlawson/pinafore/) repository, and then run the following commands in the cloned directory:
```bash
yarn # install dependencies
yarn run dev
```
The Semaphore instance will start running on `localhost:4002`.
The Pinafore instance will start running on `localhost:4002`.
To connect to the testrig, navigate to `http://localhost:4002` and enter your instance name as `localhost:8080`.

2
README.md
View File

@ -113,7 +113,7 @@ The Mastodon API has become the de facto standard for client communication with
Though most apps that implement the Mastodon API should work, GoToSocial is tested and works reliably with beautiful apps like:
* [Tusky](https://tusky.app/) for Android
* [Semaphore](https://semaphore.social/) in the browser
* [Pinafore](https://pinafore.social/) in the browser
* [Feditext](https://github.com/feditext/feditext) (beta) on iOS, iPadOS and macOS
If you've used Mastodon with a third-party app before, you'll find using GoToSocial a breeze.

22
cmd/gen-ulid/main.go Normal file
View File

@ -0,0 +1,22 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package main
import "github.com/superseriousbusiness/gotosocial/internal/id"
func main() { println(id.NewULID()) }

4
docs/admin/backup_and_restore.md
View File

@ -186,7 +186,7 @@ You'll need to put that file on your GoToSocial instance and make sure the file
For this to work reliably, you should ensure that the [storage-local-base-path](../configuration/storage.md) in your GoToSocial configuration uses an absolute path. Otherwise you'll have to tweak the paths yourself.
```sh
$ gotosocial admin media list-attachments --local-only | \
$ gotosocial --config-path /path/to/config.yaml admin media list-attachments --local-only | \
/path/to/media-to-borg-patterns.py \
<storage-local-base-path>
```
@ -210,7 +210,7 @@ If you're running Borgmatic as a systemd service, you can [create a drop-in](htt
```ini
[Service]
ExecStartPre=/path/to/gotosocial admin media list-attachments --local-only | /path/to/media-to-borg-patterns.py <storage-local-base-path> /etc/borgmatic/gotosocial_patterns
ExecStartPre=/path/to/gotosocial --config-path /path/to/config.yaml admin media list-attachments --local-only | /path/to/media-to-borg-patterns.py <storage-local-base-path> /etc/borgmatic/gotosocial_patterns
```
Documentation that's good to review:

8
docs/admin/settings.md
View File

@ -167,3 +167,11 @@ Links to the set contact account and/or email address will appear on the footer
The selected **contact user** must be an active (not suspended) admin and/or moderator on the instance.
If you're on a single-user instance and you give admin privileges to your main account, you can just fill in your own username here; you don't need to make a separate admin account just for this.
### Instance Custom CSS
custom CSS allows you to further customize the way your instance looks when visited through a browser.
This custom CSS will be applied to all pages of your instance. Users themes and CSS still take precedence over this customization.
See the [Custom CSS](./custom_css.md) page for some tips on writing custom CSS for your instance.

187
docs/api/swagger.yaml
View File

@ -1570,6 +1570,10 @@ definitions:
$ref: '#/definitions/instanceV1Configuration'
contact_account:
$ref: '#/definitions/account'
custom_css:
description: Custom CSS for the instance.
type: string
x-go-name: CustomCSS
debug:
description: Whether or not instance is running in DEBUG mode. Omitted if false.
type: boolean
@ -1750,6 +1754,10 @@ definitions:
$ref: '#/definitions/instanceV2Configuration'
contact:
$ref: '#/definitions/instanceV2Contact'
custom_css:
description: Instance Custom Css
type: string
x-go-name: CustomCSS
debug:
description: Whether or not instance is running in DEBUG mode. Omitted if false.
type: boolean
@ -2696,6 +2704,11 @@ definitions:
example: "2021-07-30T09:20:25+00:00"
type: string
x-go-name: CreatedAt
edited_at:
description: Timestamp of when the status was last edited (ISO 8601 Datetime).
example: "2021-07-30T09:20:25+00:00"
type: string
x-go-name: EditedAt
emojis:
description: Custom emoji to be used when rendering status content.
items:
@ -2893,6 +2906,11 @@ definitions:
example: "2021-07-30T09:20:25+00:00"
type: string
x-go-name: CreatedAt
edited_at:
description: Timestamp of when the status was last edited (ISO 8601 Datetime).
example: "2021-07-30T09:20:25+00:00"
type: string
x-go-name: EditedAt
emojis:
description: Custom emoji to be used when rendering status content.
items:
@ -3767,6 +3785,41 @@ paths:
summary: Block account with id.
tags:
- accounts
/api/v1/accounts/{id}/featured_tags:
get:
description: 'THIS ENDPOINT IS CURRENTLY NOT FULLY IMPLEMENTED: it will always return an empty array.'
operationId: accountsFeaturedTags
parameters:
- description: The id of the account.
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: ""
schema:
items:
type: object
type: array
"400":
description: bad request
"401":
description: unauthorized
"404":
description: not found
"406":
description: not acceptable
"500":
description: internal server error
security:
- OAuth2 Bearer:
- read:accounts
summary: Get an array of target account's featured tags.
tags:
- accounts
/api/v1/accounts/{id}/follow:
post:
consumes:
@ -6829,6 +6882,34 @@ paths:
summary: View instance rule with the given id.
tags:
- admin
/api/v1/announcements:
get:
description: 'THIS ENDPOINT IS CURRENTLY NOT FULLY IMPLEMENTED: it will always return an empty array.'
operationId: announcementsGet
produces:
- application/json
responses:
"200":
description: ""
schema:
items:
type: object
maxItems: 0
type: array
"400":
description: bad request
"401":
description: unauthorized
"406":
description: not acceptable
"500":
description: internal server error
security:
- OAuth2 Bearer:
- read:announcements
summary: Get an array of currently active announcements.
tags:
- announcements
/api/v1/apps:
post:
consumes:
@ -9859,6 +9940,112 @@ paths:
summary: Create a new status using the given form field parameters.
tags:
- statuses
put:
consumes:
- application/json
- application/x-www-form-urlencoded
description: The parameters can also be given in the body of the request, as JSON, if the content-type is set to 'application/json'.
operationId: statusEdit
parameters:
- description: |-
Text content of the status.
If media_ids is provided, this becomes optional.
Attaching a poll is optional while status is provided.
in: formData
name: status
type: string
x-go-name: Status
- description: |-
Array of Attachment ids to be attached as media.
If provided, status becomes optional, and poll cannot be used.
If the status is being submitted as a form, the key is 'media_ids[]',
but if it's json or xml, the key is 'media_ids'.
in: formData
items:
type: string
name: media_ids
type: array
x-go-name: MediaIDs
- description: |-
Array of possible poll answers.
If provided, media_ids cannot be used, and poll[expires_in] must be provided.
in: formData
items:
type: string
name: poll[options][]
type: array
x-go-name: PollOptions
- description: |-
Duration the poll should be open, in seconds.
If provided, media_ids cannot be used, and poll[options] must be provided.
format: int64
in: formData
name: poll[expires_in]
type: integer
x-go-name: PollExpiresIn
- default: false
description: Allow multiple choices on this poll.
in: formData
name: poll[multiple]
type: boolean
x-go-name: PollMultiple
- default: true
description: Hide vote counts until the poll ends.
in: formData
name: poll[hide_totals]
type: boolean
x-go-name: PollHideTotals
- description: Status and attached media should be marked as sensitive.
in: formData
name: sensitive
type: boolean
x-go-name: Sensitive
- description: |-
Text to be shown as a warning or subject before the actual content.
Statuses are generally collapsed behind this field.
in: formData
name: spoiler_text
type: string
x-go-name: SpoilerText
- description: ISO 639 language code for this status.
in: formData
name: language
type: string
x-go-name: Language
- description: Content type to use when parsing this status.
enum:
- text/plain
- text/markdown
in: formData
name: content_type
type: string
x-go-name: ContentType
produces:
- application/json
responses:
"200":
description: The latest status revision.
schema:
$ref: '#/definitions/status'
"400":
description: bad request
"401":
description: unauthorized
"403":
description: forbidden
"404":
description: not found
"406":
description: not acceptable
"500":
description: internal server error
security:
- OAuth2 Bearer:
- write:statuses
summary: Edit an existing status using the given form field parameters.
tags:
- statuses
/api/v1/statuses/{id}:
delete:
description: |-

2
docs/faq.md
View File

@ -2,7 +2,7 @@
## Where's the user interface?
GoToSocial is just a bare server for the most part and is designed to be used thru external applications. [Semaphore](https://semaphore.social/) in the browser, [Tusky](https://tusky.app/) for Android and [Feditext](https://github.com/feditext/feditext) for iOS, iPadOS and macOS are the best-supported. Anything that supports the Mastodon API should work, other than the features GoToSocial doesn't yet have. Permalinks and profile pages are served directly through GoToSocial as well as the settings panel, but most interaction goes through the apps.
GoToSocial is just a bare server for the most part and is designed to be used thru external applications. [Pinafore](https://pinafore.social/) in the browser, [Tusky](https://tusky.app/) for Android and [Feditext](https://github.com/feditext/feditext) for iOS, iPadOS and macOS are the best-supported. Anything that supports the Mastodon API should work, other than the features GoToSocial doesn't yet have. Permalinks and profile pages are served directly through GoToSocial as well as the settings panel, but most interaction goes through the apps.
## Why aren't my posts showing up on my profile page?

2
docs/getting_started/reverse_proxy/websocket.md
View File

@ -1,6 +1,6 @@
# WebSocket
GoToSocial uses the secure [WebSocket protocol](https://en.wikipedia.org/wiki/WebSocket) (aka `wss`) to allow for streaming updates of statuses and notifications via client apps like Semaphore.
GoToSocial uses the secure [WebSocket protocol](https://en.wikipedia.org/wiki/WebSocket) (aka `wss`) to allow for streaming updates of statuses and notifications via client apps like Pinafore.
In order to use this functionality, you need to ensure that whatever proxy you've configured GoToSocial to run behind allows WebSocket connections through.

415
docs/locales/zh/api/swagger.yaml
View File

@ -4980,7 +4980,7 @@ paths:
- description: 此表情的代码，将被实例居民用于选定对应表情。此代码在实例上必须是唯一的。
in: formData
name: shortcode
pattern: \w{2,30}
pattern: \w{1,30}
required: true
type: string
- description: 此表情的 png 或 gif 图像。动画 png 也可以！为了确保与其他 fedi 实现的兼容性，默认情况下表情大小限制为 50kb。
@ -5130,7 +5130,7 @@ paths:
- description: 用于表情的代码，将被实例居民用于选定表情。此代码在实例上必须是唯一的。仅适用于 `copy` 操作类型。
in: formData
name: shortcode
pattern: \w{2,30}
pattern: \w{1,30}
type: string
- description: 此表情的新 png 或 gif 图像。动画 png 也可以！为了确保与其他 fedi 实现的兼容性，默认情况下表情大小限制为 50kb。仅适用于 **本站** 表情。
in: formData
@ -5639,6 +5639,417 @@ paths:
summary: 吊销实例密钥
tags:
- admin
/api/v1/admin/domain_permission_drafts:
get:
description: |-
该端点将返回按时间倒序排序（最新优先），并带有连续 ID 的域名权限草案（ID 值越大，草稿越新）。可以通过返回的 Link 标头解析下一页与上一页查询。
示例:
```
<https://example.org/api/v1/admin/domain_permission_drafts?limit=20&max_id=01FC0SKA48HNSVR6YKZCQGS2V8>; rel="next", <https://example.org/api/v1/admin/domain_permission_drafts?limit=20&min_id=01FC0SKW5JK2Q4EVAV2B462YY0>; rel="prev"
````
operationId: domainPermissionDraftsGet
parameters:
- description: 仅显示给定订阅 ID 创建的草案。
in: query
name: subscription_id
type: string
- description: 仅显示针对特定域名的草案。
in: query
name: domain
type: string
- description: 筛选“屏蔽”与“放行”类型的草案。
in: query
name: permission_type
type: string
- description: 仅返回早于给定 max ID 的条目（用于向下分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: max_id
type: string
- description: 仅返回晚于给定 since ID 的条目（用于向上分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: since_id
type: string
- description: 仅返回相邻且晚于给定 min ID 的条目（用于向上分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: min_id
type: string
- default: 20
description: 要返回的条目数量。
in: query
maximum: 100
minimum: 1
name: limit
type: integer
produces:
- application/json
responses:
"200":
description: 域名权限草案。
headers:
Link:
description: 下一查询与上一查询的链接。
type: string
schema:
items:
$ref: '#/definitions/domainPermission'
type: array
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止
"404":
description: not found 未找到
"406":
description: not acceptable 不可接受
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 查看域名权限草案。
tags:
- admin
post:
consumes:
- multipart/form-data
- application/json
operationId: domainPermissionDraftCreate
parameters:
- description: 该草案要针对的域名。
in: formData
name: domain
type: string
- description: 草案类型为“放行”或“屏蔽”。
in: formData
name: permission_type
type: string
- description: 对外公开展示时混淆具体域名。例如：`example.org` 将变为类似 `ex***e.org` 的字符串。
in: formData
name: obfuscate
type: boolean
- description: 对此域名权限的公开评注。若您选择分享此权限设定，此评注将与权限条目一起显示。
in: formData
name: public_comment
type: string
- description: 对此域名权限的私人评注。仅显示给其他管理员，因此这是一个可用于记录为什么某个域名最终被添加此权限设定的有用的内部手段。
in: formData
name: private_comment
type: string
produces:
- application/json
responses:
"200":
description: 新创建的域名权限草案。
schema:
$ref: '#/definitions/domainPermission'
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"406":
description: not acceptable 不可接受
"409":
description: conflict 冲突
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 使用给定参数创建一条域名权限草案。
tags:
- admin
/api/v1/admin/domain_permission_drafts/{id}:
get:
operationId: domainPermissionDraftGet
parameters:
- description: 域名权限草案的 ID。
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: 域名权限草案。
schema:
$ref: '#/definitions/domainPermission'
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"404":
description: not found 未找到
"406":
description: not acceptable 不可接受
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 获取具有给定 ID 的域名权限草案。
tags:
- admin
/api/v1/admin/domain_permission_drafts/{id}/accept:
post:
consumes:
- multipart/form-data
- application/json
operationId: domainPermissionDraftAccept
parameters:
- description: 域名权限草案的 ID。
in: path
name: id
required: true
type: string
- default: false
description: 若已经存在一条具有相同域名与权限设定类型的草案，使用新草案的字段覆盖现有权限设定。
in: formData
name: overwrite
type: boolean
produces:
- application/json
responses:
"200":
description: 新创建的域名权限草案。
schema:
$ref: '#/definitions/domainPermission'
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"406":
description: not acceptable 不可接受
"409":
description: conflict 冲突
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 接受一条域名权限草案，将其转换为会得到强制执行的域名权限。
tags:
- admin
/api/v1/admin/domain_permission_drafts/{id}/remove:
post:
consumes:
- multipart/form-data
- application/json
operationId: domainPermissionDraftRemove
parameters:
- description: 域名权限草案的 ID。
in: path
name: id
required: true
type: string
- default: false
description: 删除此域名权限草案时，为目标域名创建一个域名排除条目，以确保之后不会为此域名创建草案。
in: formData
name: exclude_target
type: boolean
produces:
- application/json
responses:
"200":
description: 被移除的域名权限草案。
schema:
$ref: '#/definitions/domainPermission'
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"406":
description: not acceptable 不可接受
"409":
description: conflict 冲突
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 移除一条域名权限草案，可选择忽略所有之后的针对给定域名的草案。
tags:
- admin
/api/v1/admin/domain_permission_excludes:
get:
description: |-
返回按时间倒序排序（新创建的条目优先），并带有连续 ID 的域名权限排除条目（ID 值越大，排除条目越新）。可以通过返回的 Link 标头解析下一页与上一页查询。
示例：
```
<https://example.org/api/v1/admin/domain_permission_excludes?limit=20&max_id=01FC0SKA48HNSVR6YKZCQGS2V8>; rel="next", <https://example.org/api/v1/admin/domain_permission_excludes?limit=20&min_id=01FC0SKW5JK2Q4EVAV2B462YY0>; rel="prev"
```
operationId: domainPermissionExcludesGet
parameters:
- description: 仅返回针对给定域名的排除条目。
in: query
name: domain
type: string
- description: 仅返回比给定 max ID 新的条目（用于向下分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: max_id
type: string
- description: 仅返回比给定 since ID 新的条目（用于向上分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: since_id
type: string
- description: 仅返回比给定 min ID 相邻且更新的条目（用于向上分页）。具有对应 ID 的条目不会包含在响应中。
in: query
name: min_id
type: string
- default: 20
description: 要返回的条目数量。
in: query
maximum: 100
minimum: 1
name: limit
type: integer
produces:
- application/json
responses:
"200":
description: 域名权限排除条目。
headers:
Link:
description: 下一查询与上一查询的链接。
type: string
schema:
items:
$ref: '#/definitions/domainPermission'
type: array
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"404":
description: not found 未找到
"406":
description: not acceptable 不可接受
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 查看域名权限排除条目。
tags:
- admin
post:
consumes:
- multipart/form-data
- application/json
description: |-
被排除的域名（及其子域名）在导入或订阅域名权限列表时不会被自动屏蔽或放行。
您仍然可以为被排除的域名手动创建域名屏蔽条目或域名放行条目，被排除之后，与该域名关联的任何的已有或新创建的域名屏蔽条目或域名放行条目都将被继续执行。
operationId: domainPermissionExcludeCreate
parameters:
- description: 要创建权限排除的域名。
in: formData
name: domain
type: string
- description: 对该域名排除条目的私密评论。
in: formData
name: private_comment
type: string
produces:
- application/json
responses:
"200":
description: 新创建的域名排除条目。
schema:
$ref: '#/definitions/domainPermission'
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"406":
description: not acceptable 不可接受
"409":
description: conflict 冲突
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 使用给定参数创建一个域名权限排除条目。
tags:
- admin
/api/v1/admin/domain_permission_excludes/{id}:
delete:
operationId: domainPermissionExcludeDelete
parameters:
- description: 该域名权限排除条目的 ID。
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: 被移除的域名权限排除条目。
schema:
$ref: '#/definitions/domainPermission'
"400":
description: bad request 无效请求
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"406":
description: not acceptable 不可接受
"409":
description: conflict 冲突
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 移除一个域名权限排除条目。
tags:
- admin
get:
operationId: domainPermissionExcludeGet
parameters:
- description: 域名权限排除条目的 ID。
in: path
name: id
required: true
type: string
produces:
- application/json
responses:
"200":
description: 域名权限排除条目。
schema:
$ref: '#/definitions/domainPermission'
"401":
description: unauthorized 未授权
"403":
description: forbidden 禁止访问
"404":
description: not found 未找到
"406":
description: not acceptable 不可接受
"500":
description: internal server error 服务器内部错误
security:
- OAuth2 Bearer:
- admin
summary: 获取具有给定 ID 的域名权限排除。
tags:
- admin
/api/v1/admin/email/test:
post:
consumes:

4
docs/locales/zh/configuration/storage.md
View File

@ -1,5 +1,9 @@
# 存储
When configuring an object storage backend, the `storage-s3-endpoint` **must not** include the bucket name. That's what `s3-bucket-name` is for. Using subfolders in a bucket isn't currently supported.
配置对象存储后端时，`storage-s3-endpoint` **不得** 包含存储桶名称。`s3-bucket-name`负责配置存储桶名称。目前不支持使用特定存储桶的子目录作为存储后端。
## 设置
```yaml

2
docs/locales/zh/faq.md
View File

@ -2,7 +2,7 @@
## 用户界面在哪？
GoToSocial 的大部分内容只是一个裸服务端，用于通过第三方应用程序进行使用。可通过 [Semaphore](https://semaphore.social/) 在浏览器使用，可通过 [Tusky](https://tusky.app/) 在 Android 使用，可通过 [Feditext](https://github.com/feditext/feditext) 在 iOS、iPadOS 和 macOS 使用。这些应用程序兼容性最好。任何由 Mastodon API 提供的实例功能都应该可以工作，除非它们是 GoToSocial 尚不具备的功能。永久链接和个账户页是通过 GoToSocial 直接提供的，设置面板也是如此，但大多数交互都是通过应用程序完成的。
GoToSocial 的大部分内容只是一个裸服务端，用于通过第三方应用程序进行使用。可通过 [Pinafore](https://pinafore.social/) 在浏览器使用，可通过 [Tusky](https://tusky.app/) 在 Android 使用，可通过 [Feditext](https://github.com/feditext/feditext) 在 iOS、iPadOS 和 macOS 使用。这些应用程序兼容性最好。任何由 Mastodon API 提供的实例功能都应该可以工作，除非它们是 GoToSocial 尚不具备的功能。永久链接和个账户页是通过 GoToSocial 直接提供的，设置面板也是如此，但大多数交互都是通过应用程序完成的。
## 为什么我的贴文没有显示在我的账户页面上？

2
docs/locales/zh/getting_started/reverse_proxy/websocket.md
View File

@ -1,6 +1,6 @@
# WebSocket
GoToSocial 使用安全的 [WebSocket 协议](https://en.wikipedia.org/wiki/WebSocket)（即 `wss`）来通过客户端应用程序（如 Semaphore）实现贴文和通知的实时更新。
GoToSocial 使用安全的 [WebSocket 协议](https://en.wikipedia.org/wiki/WebSocket)（即 `wss`）来通过客户端应用程序（如 Pinafore）实现贴文和通知的实时更新。
为了使用此功能，你需要确保配置 GoToSocial 所在的代理允许 WebSocket 连接通过。

10
docs/locales/zh/repo/CONTRIBUTING.md
View File

@ -24,7 +24,7 @@
- [浏览代码结构](#浏览代码结构)
- [风格/代码检查/格式化](#风格代码检查格式化)
- [测试](#测试)
- [独立测试环境与 Semaphore](#独立测试环境与-semaphore)
- [独立测试环境与 Pinafore](#独立测试环境与-pinafore)
- [运行自动化测试](#运行自动化测试)
- [SQLite](#sqlite)
- [Postgres](#postgres)
@ -400,9 +400,9 @@ GoToSocial 提供了一个 [testrig](https://github.com/superseriousbusiness/got
没有模拟的一个东西是数据库接口，因为使用内存中的 SQLite 数据库比模拟所有东西要简单得多。
#### 独立测试环境与 Semaphore
#### 独立测试环境与 Pinafore
你可以启动一个在本地主机运行的独立测试服务器 testrig，可以通过 [Semaphore](https://github.com/NickColley/semaphore/) 连接。
你可以启动一个在本地主机运行的独立测试服务器 testrig，可以通过 [Pinafore](https://github.com/NickColley/pinafore/) 连接。
要做到这一点，首先用 `DEBUG=1 ./scripts/build.sh` 构建 gotosocial 二进制文件。
@ -412,14 +412,14 @@ GoToSocial 提供了一个 [testrig](https://github.com/superseriousbusiness/got
DEBUG=1 ./gotosocial testrig start
```
要在本地开发模式下运行 Semaphore，首先克隆 [Semaphore](https://github.com/NickColley/semaphore/) 存储库，然后在克隆的目录中运行以下命令：
要在本地开发模式下运行 Pinafore，首先克隆 [Pinafore](https://github.com/nolanlawson/pinafore/) 存储库，然后在克隆的目录中运行以下命令：
```bash
yarn # 安装依赖
yarn run dev
```
Semaphore 实例将在 `localhost:4002` 上启动。
Pinafore 实例将在 `localhost:4002` 上启动。
要连接到 testrig，导航至 `http://localhost:4002`，并将在实例域名栏输入 `localhost:8080`。

2
docs/locales/zh/repo/README.md
View File

@ -113,7 +113,7 @@ Mastodon API 已成为客户端与联邦宇宙服务端通信的事实标准，
大多数实现 Mastodon API 的应用程序都应该可以使用 GoToSocial，但以下这些优秀的应用程序已经过测试，可与 GoToSocial 可靠地配合使用：
* [Tusky](https://tusky.app/) 适用于 Android
* [Semaphore](https://semaphore.social/) 适用于浏览器
* [Pinafore](https://pinafore.social/) 适用于浏览器
* [Feditext](https://github.com/feditext/feditext) (beta) 适用于 iOS, iPadOS 和 macOS
如果你之前通过第三方应用来使用 Mastodon，使用 GoToSocial 将是轻而易举的。

3
docs/locales/zh/user_guide/posts.md
View File

@ -36,6 +36,9 @@ GoToSocial 为贴文提供 Mastodon 风格的隐私设置。从最私密到最
### 互关可见
!!! warning
目前暂时无法将帖文可见性设为“互关可见”。
`互关可见` 的贴文只会显示给贴文作者和与作者*互相关注*的人。换句话说，只有在满足两个条件时，其他人才能看到：
1. 其他账户关注贴文作者。

BIN
docs/overrides/public/admin-settings-instance.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 200 KiB

After

Width:  |  Height:  |  Size: 229 KiB

3
docs/user_guide/posts.md
View File

@ -285,6 +285,9 @@ For accessibility reasons, it is considerate to use upper camel case when you're
You can include as many hashtags as you like within a GoToSocial post, and each hashtag has a length limit of 100 characters.
!!! tip
To end a hashtag, you can simply use a space, for example in the text `this #soup rules`, the hashtag is terminated by a space so `#soup` becomes the hashtag. However, you can also use a pipe character `|`, or the unicode characters `\u200B` (zero-width no-break space) or `\uFEFF` (zero-width space), to create "partial-word" hashtags. For example, with input text `this #so|up rules`, only the `#so` part becomes the hashtag. Likewise, with the input text `this #so​up rules`, which contains an invisible zero-width space after the o and before the u, only the `#so` part becomes the hashtag. See here for more information on zero-width spaces: https://en.wikipedia.org/wiki/Zero-width_space.
## Input Sanitization
In order not to spread scripts, vulnerabilities, and glitchy HTML all over the place, GoToSocial performs the following types of input sanitization:

22
go.mod
View File

@ -6,7 +6,7 @@ go 1.23
replace github.com/go-swagger/go-swagger => github.com/superseriousbusiness/go-swagger v0.31.0-gts-go1.23-fix
// Replace modernc/sqlite with our version that fixes the concurrency INTERRUPT issue
replace modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.33.1-concurrency-workaround
replace modernc.org/sqlite => gitlab.com/NyaaaWhatsUpDoc/sqlite v1.34.2-concurrency-workaround
// Below pin otel libraries to v1.29.0 until we can figure out issues
replace go.opentelemetry.io/otel => go.opentelemetry.io/otel v1.29.0
@ -31,7 +31,7 @@ require (
codeberg.org/gruf/go-debug v1.3.0
codeberg.org/gruf/go-errors/v2 v2.3.2
codeberg.org/gruf/go-fastcopy v1.1.3
codeberg.org/gruf/go-ffmpreg v0.6.0
codeberg.org/gruf/go-ffmpreg v0.6.4
codeberg.org/gruf/go-iotools v0.0.0-20240710125620-934ae9c654cf
codeberg.org/gruf/go-kv v1.6.5
codeberg.org/gruf/go-list v0.0.0-20240425093752-494db03d641f
@ -44,6 +44,7 @@ require (
codeberg.org/superseriousbusiness/exif-terminator v0.9.1
github.com/DmitriyVTitov/size v1.5.0
github.com/KimMachineGun/automemlimit v0.6.1
github.com/SherClockHolmes/webpush-go v1.3.0
github.com/buckket/go-blurhash v1.1.0
github.com/coreos/go-oidc/v3 v3.11.0
github.com/gin-contrib/cors v1.7.2
@ -60,12 +61,11 @@ require (
github.com/k3a/html2text v1.2.1
github.com/microcosm-cc/bluemonday v1.0.27
github.com/miekg/dns v1.1.62
github.com/minio/minio-go/v7 v7.0.80
github.com/minio/minio-go/v7 v7.0.81
github.com/mitchellh/mapstructure v1.5.0
github.com/ncruces/go-sqlite3 v0.20.3
github.com/ncruces/go-sqlite3 v0.21.3
github.com/oklog/ulid v1.3.1
github.com/prometheus/client_golang v1.20.5
github.com/SherClockHolmes/webpush-go v1.3.0
github.com/spf13/cobra v1.8.1
github.com/spf13/viper v1.19.0
github.com/stretchr/testify v1.10.0
@ -92,11 +92,11 @@ require (
go.opentelemetry.io/otel/sdk/metric v1.32.0
go.opentelemetry.io/otel/trace v1.32.0
go.uber.org/automaxprocs v1.6.0
golang.org/x/crypto v0.29.0
golang.org/x/image v0.22.0
golang.org/x/net v0.31.0
golang.org/x/crypto v0.31.0
golang.org/x/image v0.23.0
golang.org/x/net v0.32.0
golang.org/x/oauth2 v0.24.0
golang.org/x/text v0.20.0
golang.org/x/text v0.21.0
gopkg.in/mcuadros/go-syslog.v2 v2.3.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v0.0.0-00010101000000-000000000000
@ -235,8 +235,8 @@ require (
golang.org/x/arch v0.8.0 // indirect
golang.org/x/exp v0.0.0-20240222234643-814bf88cf225 // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/sync v0.9.0 // indirect
golang.org/x/sys v0.27.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/tools v0.22.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect

9
internal/ap/interfaces.go
View File

@ -25,8 +25,11 @@
// IsActivityable returns whether AS vocab type name is acceptable as Activityable.
func IsActivityable(typeName string) bool {
return isActivity(typeName) ||
isIntransitiveActivity(typeName)
return isActivity(typeName)
// See interfaces_test.go comment
// about intransitive activities:
//
// || isIntransitiveActivity(typeName)
}
// ToActivityable safely tries to cast vocab.Type as Activityable, also checking for expected AS type names.
@ -184,6 +187,7 @@ type Accountable interface {
WithEndpoints
WithTag
WithPublished
WithUpdated
}
// Statusable represents the minimum activitypub interface for representing a 'status'.
@ -196,6 +200,7 @@ type Statusable interface {
WithName
WithInReplyTo
WithPublished
WithUpdated
WithURL
WithAttributedTo
WithTo

93
internal/ap/interfaces_test.go Normal file
View File

@ -0,0 +1,93 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package ap_test
import (
"github.com/superseriousbusiness/activity/streams/vocab"
"github.com/superseriousbusiness/gotosocial/internal/ap"
)
var (
// NOTE: the below aren't actually tests that are run,
// we just move them into an _test.go file to declutter
// the main interfaces.go file, which is already long.
// Compile-time checks for Activityable interface methods.
_ ap.Activityable = (vocab.ActivityStreamsAccept)(nil)
_ ap.Activityable = (vocab.ActivityStreamsTentativeAccept)(nil)
_ ap.Activityable = (vocab.ActivityStreamsAdd)(nil)
_ ap.Activityable = (vocab.ActivityStreamsCreate)(nil)
_ ap.Activityable = (vocab.ActivityStreamsDelete)(nil)
_ ap.Activityable = (vocab.ActivityStreamsFollow)(nil)
_ ap.Activityable = (vocab.ActivityStreamsIgnore)(nil)
_ ap.Activityable = (vocab.ActivityStreamsJoin)(nil)
_ ap.Activityable = (vocab.ActivityStreamsLeave)(nil)
_ ap.Activityable = (vocab.ActivityStreamsLike)(nil)
_ ap.Activityable = (vocab.ActivityStreamsOffer)(nil)
_ ap.Activityable = (vocab.ActivityStreamsInvite)(nil)
_ ap.Activityable = (vocab.ActivityStreamsReject)(nil)
_ ap.Activityable = (vocab.ActivityStreamsTentativeReject)(nil)
_ ap.Activityable = (vocab.ActivityStreamsRemove)(nil)
_ ap.Activityable = (vocab.ActivityStreamsUndo)(nil)
_ ap.Activityable = (vocab.ActivityStreamsUpdate)(nil)
_ ap.Activityable = (vocab.ActivityStreamsView)(nil)
_ ap.Activityable = (vocab.ActivityStreamsListen)(nil)
_ ap.Activityable = (vocab.ActivityStreamsRead)(nil)
_ ap.Activityable = (vocab.ActivityStreamsMove)(nil)
_ ap.Activityable = (vocab.ActivityStreamsAnnounce)(nil)
_ ap.Activityable = (vocab.ActivityStreamsBlock)(nil)
_ ap.Activityable = (vocab.ActivityStreamsFlag)(nil)
_ ap.Activityable = (vocab.ActivityStreamsDislike)(nil)
// the below intransitive activities don't fit the interface definition because they're
// missing an attached object (as the activity itself contains the details), but we don't
// actually end up using them so it's simpler to just comment them out and not have to do
// a WithObject{} interface check on every single incoming activity:
//
// _ Activityable = (vocab.ActivityStreamsArrive)(nil)
// _ Activityable = (vocab.ActivityStreamsTravel)(nil)
// _ Activityable = (vocab.ActivityStreamsQuestion)(nil)
// Compile-time checks for Accountable interface methods.
_ ap.Accountable = (vocab.ActivityStreamsPerson)(nil)
_ ap.Accountable = (vocab.ActivityStreamsApplication)(nil)
_ ap.Accountable = (vocab.ActivityStreamsOrganization)(nil)
_ ap.Accountable = (vocab.ActivityStreamsService)(nil)
_ ap.Accountable = (vocab.ActivityStreamsGroup)(nil)
// Compile-time checks for Statusable interface methods.
_ ap.Statusable = (vocab.ActivityStreamsArticle)(nil)
_ ap.Statusable = (vocab.ActivityStreamsDocument)(nil)
_ ap.Statusable = (vocab.ActivityStreamsImage)(nil)
_ ap.Statusable = (vocab.ActivityStreamsVideo)(nil)
_ ap.Statusable = (vocab.ActivityStreamsNote)(nil)
_ ap.Statusable = (vocab.ActivityStreamsPage)(nil)
_ ap.Statusable = (vocab.ActivityStreamsEvent)(nil)
_ ap.Statusable = (vocab.ActivityStreamsPlace)(nil)
_ ap.Statusable = (vocab.ActivityStreamsProfile)(nil)
_ ap.Statusable = (vocab.ActivityStreamsQuestion)(nil)
// Compile-time checks for Pollable interface methods.
_ ap.Pollable = (vocab.ActivityStreamsQuestion)(nil)
// Compile-time checks for PollOptionable interface methods.
_ ap.PollOptionable = (vocab.ActivityStreamsNote)(nil)
// Compile-time checks for Acceptable interface methods.
_ ap.Acceptable = (vocab.ActivityStreamsAccept)(nil)
)

19
internal/ap/properties.go
View File

@ -408,6 +408,25 @@ func SetPublished(with WithPublished, published time.Time) {
publishProp.Set(published)
}
// GetUpdated returns the time contained in the Updated property of 'with'.
func GetUpdated(with WithUpdated) time.Time {
updateProp := with.GetActivityStreamsUpdated()
if updateProp == nil || !updateProp.IsXMLSchemaDateTime() {
return time.Time{}
}
return updateProp.Get()
}
// SetUpdated sets the given time on the Updated property of 'with'.
func SetUpdated(with WithUpdated, updated time.Time) {
updateProp := with.GetActivityStreamsUpdated()
if updateProp == nil {
updateProp = streams.NewActivityStreamsUpdatedProperty()
with.SetActivityStreamsUpdated(updateProp)
}
updateProp.Set(updated)
}
// GetEndTime returns the time contained in the EndTime property of 'with'.
func GetEndTime(with WithEndTime) time.Time {
endTimeProp := with.GetActivityStreamsEndTime()

16
internal/api/activitypub/users/outboxget_test.go
View File

@ -82,7 +82,7 @@ func (suite *OutboxGetTestSuite) TestGetOutbox() {
"@context": "https://www.w3.org/ns/activitystreams",
"first": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40",
"id": "http://localhost:8080/users/the_mighty_zork/outbox",
"totalItems": 8,
"totalItems": 9,
"type": "OrderedCollection"
}`, dst.String())
@ -142,6 +142,14 @@ func (suite *OutboxGetTestSuite) TestGetOutboxFirstPage() {
"id": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40",
"next": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40\u0026max_id=01F8MHAMCHF6Y650WCRSCP4WMY",
"orderedItems": [
{
"actor": "http://localhost:8080/users/the_mighty_zork",
"cc": "http://localhost:8080/users/the_mighty_zork/followers",
"id": "http://localhost:8080/users/the_mighty_zork/statuses/01JDPZC707CKDN8N4QVWM4Z1NR/activity#Create",
"object": "http://localhost:8080/users/the_mighty_zork/statuses/01JDPZC707CKDN8N4QVWM4Z1NR",
"to": "https://www.w3.org/ns/activitystreams#Public",
"type": "Create"
},
{
"actor": "http://localhost:8080/users/the_mighty_zork",
"cc": "http://localhost:8080/users/the_mighty_zork/followers",
@ -160,8 +168,8 @@ func (suite *OutboxGetTestSuite) TestGetOutboxFirstPage() {
}
],
"partOf": "http://localhost:8080/users/the_mighty_zork/outbox",
"prev": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40\u0026min_id=01HH9KYNQPA416TNJ53NSATP40",
"totalItems": 8,
"prev": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40\u0026min_id=01JDPZC707CKDN8N4QVWM4Z1NR",
"totalItems": 9,
"type": "OrderedCollectionPage"
}`, dst.String())
@ -224,7 +232,7 @@ func (suite *OutboxGetTestSuite) TestGetOutboxNextPage() {
"id": "http://localhost:8080/users/the_mighty_zork/outbox?limit=40&max_id=01F8MHAMCHF6Y650WCRSCP4WMY",
"orderedItems": [],
"partOf": "http://localhost:8080/users/the_mighty_zork/outbox",
"totalItems": 8,
"totalItems": 9,
"type": "OrderedCollectionPage"
}`, dst.String())

4
internal/api/client.go
View File

@ -23,6 +23,7 @@
"github.com/gin-gonic/gin"
"github.com/superseriousbusiness/gotosocial/internal/api/client/accounts"
"github.com/superseriousbusiness/gotosocial/internal/api/client/admin"
"github.com/superseriousbusiness/gotosocial/internal/api/client/announcements"
"github.com/superseriousbusiness/gotosocial/internal/api/client/apps"
"github.com/superseriousbusiness/gotosocial/internal/api/client/blocks"
"github.com/superseriousbusiness/gotosocial/internal/api/client/bookmarks"
@ -67,6 +68,7 @@ type Client struct {
accounts *accounts.Module // api/v1/accounts, api/v1/profile
admin *admin.Module // api/v1/admin
announcements *announcements.Module // api/v1/announcements
apps *apps.Module // api/v1/apps
blocks *blocks.Module // api/v1/blocks
bookmarks *bookmarks.Module // api/v1/bookmarks
@ -119,6 +121,7 @@ func (c *Client) Route(r *router.Router, m ...gin.HandlerFunc) {
h := apiGroup.Handle
c.accounts.Route(h)
c.admin.Route(h)
c.announcements.Route(h)
c.apps.Route(h)
c.blocks.Route(h)
c.bookmarks.Route(h)
@ -159,6 +162,7 @@ func NewClient(state *state.State, p *processing.Processor) *Client {
accounts: accounts.New(p),
admin: admin.New(state, p),
announcements: announcements.New(p),
apps: apps.New(p),
blocks: blocks.New(p),
bookmarks: bookmarks.New(p),

4
internal/api/client/accounts/accounts.go
View File

@ -40,6 +40,7 @@
BlockPath = BasePathWithID + "/block"
DeletePath = BasePath + "/delete"
FeaturedTagsPath = BasePathWithID + "/featured_tags"
FollowersPath = BasePathWithID + "/followers"
FollowingPath = BasePathWithID + "/following"
FollowPath = BasePathWithID + "/follow"
@ -98,6 +99,9 @@ func (m *Module) Route(attachHandler func(method string, path string, f ...gin.H
// get account's statuses
attachHandler(http.MethodGet, StatusesPath, m.AccountStatusesGETHandler)
// get account's featured tags
attachHandler(http.MethodGet, FeaturedTagsPath, m.AccountFeaturedTagsGETHandler)
// get following or followers
attachHandler(http.MethodGet, FollowersPath, m.AccountFollowersGETHandler)
attachHandler(http.MethodGet, FollowingPath, m.AccountFollowingGETHandler)

2
internal/api/client/accounts/accountverify_test.go
View File

@ -97,7 +97,7 @@ func (suite *AccountVerifyTestSuite) TestAccountVerifyGet() {
suite.Equal("http://localhost:8080/fileserver/01F8MH1H7YV1Z7D2C8K2730QBF/header/small/01PFPMWK2FF0D9WMHEJHR07C3Q.webp", apimodelAccount.HeaderStatic)
suite.Equal(2, apimodelAccount.FollowersCount)
suite.Equal(2, apimodelAccount.FollowingCount)
suite.Equal(8, apimodelAccount.StatusesCount)
suite.Equal(9, apimodelAccount.StatusesCount)
suite.EqualValues(apimodel.VisibilityPublic, apimodelAccount.Source.Privacy)
suite.Equal(testAccount.Settings.Language, apimodelAccount.Source.Language)
suite.Equal(testAccount.NoteRaw, apimodelAccount.Source.Note)

83
internal/api/client/accounts/featuredtags.go Normal file
View File

@ -0,0 +1,83 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package accounts
import (
"net/http"
"github.com/gin-gonic/gin"
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/oauth"
)
// AccountFeaturedTagsGETHandler swagger:operation GET /api/v1/accounts/{id}/featured_tags accountsFeaturedTags
//
// Get an array of target account's featured tags.
//
// THIS ENDPOINT IS CURRENTLY NOT FULLY IMPLEMENTED: it will always return an empty array.
//
// ---
// tags:
// - accounts
//
// produces:
// - application/json
//
// parameters:
// -
// name: id
// type: string
// description: The id of the account.
// in: path
// required: true
//
// security:
// - OAuth2 Bearer:
// - read:accounts
//
// responses:
// '200':
// schema:
// type: array
// items:
// type: object
// '400':
// description: bad request
// '401':
// description: unauthorized
// '404':
// description: not found
// '406':
// description: not acceptable
// '500':
// description: internal server error
func (m *Module) AccountFeaturedTagsGETHandler(c *gin.Context) {
_, err := oauth.Authed(c, true, true, true, true)
if err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorUnauthorized(err, err.Error()), m.processor.InstanceGetV1)
return
}
if _, err := apiutil.NegotiateAccept(c, apiutil.JSONAcceptHeaders...); err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorNotAcceptable(err, err.Error()), m.processor.InstanceGetV1)
return
}
apiutil.Data(c, http.StatusOK, apiutil.AppJSON, apiutil.EmptyJSONArray)
}

12
internal/api/client/admin/accountsgetv2_test.go
View File

@ -99,8 +99,8 @@ func (suite *AccountsGetTestSuite) TestAccountsGetFromTop() {
"header_description": "Flat gray background (default header).",
"followers_count": 1,
"following_count": 1,
"statuses_count": 8,
"last_status_at": "2021-07-28",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [
{
@ -262,8 +262,8 @@ func (suite *AccountsGetTestSuite) TestAccountsGetFromTop() {
"header_media_id": "01PFPMWK2FF0D9WMHEJHR07C3Q",
"followers_count": 2,
"following_count": 2,
"statuses_count": 8,
"last_status_at": "2024-01-10",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [],
"enable_rss": true
@ -403,8 +403,8 @@ func (suite *AccountsGetTestSuite) TestAccountsGetFromTop() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}

47
internal/api/client/admin/reportsget_test.go
View File

@ -186,8 +186,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -232,8 +232,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
"header_description": "Flat gray background (default header).",
"followers_count": 1,
"following_count": 1,
"statuses_count": 8,
"last_status_at": "2021-07-28",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [
{
@ -414,8 +414,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
"header_description": "Flat gray background (default header).",
"followers_count": 1,
"following_count": 1,
"statuses_count": 8,
"last_status_at": "2021-07-28",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [
{
@ -473,8 +473,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -485,6 +485,7 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
{
"id": "01FVW7JHQFSFK166WWKR8CBA6M",
"created_at": "2021-09-20T10:40:37.000Z",
"edited_at": null,
"in_reply_to_id": null,
"in_reply_to_account_id": null,
"sensitive": false,
@ -521,8 +522,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetAll() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
},
@ -667,8 +668,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetCreatedByAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 1,
"following_count": 1,
"statuses_count": 8,
"last_status_at": "2021-07-28",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [
{
@ -726,8 +727,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetCreatedByAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -738,6 +739,7 @@ func (suite *ReportsGetTestSuite) TestReportsGetCreatedByAccount() {
{
"id": "01FVW7JHQFSFK166WWKR8CBA6M",
"created_at": "2021-09-20T10:40:37.000Z",
"edited_at": null,
"in_reply_to_id": null,
"in_reply_to_account_id": null,
"sensitive": false,
@ -774,8 +776,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetCreatedByAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
},
@ -920,8 +922,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetTargetAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 1,
"following_count": 1,
"statuses_count": 8,
"last_status_at": "2021-07-28",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [
{
@ -979,8 +981,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetTargetAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -991,6 +993,7 @@ func (suite *ReportsGetTestSuite) TestReportsGetTargetAccount() {
{
"id": "01FVW7JHQFSFK166WWKR8CBA6M",
"created_at": "2021-09-20T10:40:37.000Z",
"edited_at": null,
"in_reply_to_id": null,
"in_reply_to_account_id": null,
"sensitive": false,
@ -1027,8 +1030,8 @@ func (suite *ReportsGetTestSuite) TestReportsGetTargetAccount() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
},

42
internal/api/client/announcements/announcements.go Normal file
View File

@ -0,0 +1,42 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package announcements
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/superseriousbusiness/gotosocial/internal/processing"
)
// BasePath is the base path for this api module, excluding the api prefix
const BasePath = "/v1/announcements"
type Module struct {
processor *processing.Processor
}
func New(processor *processing.Processor) *Module {
return &Module{
processor: processor,
}
}
func (m *Module) Route(attachHandler func(method string, path string, f ...gin.HandlerFunc) gin.IRoutes) {
attachHandler(http.MethodGet, BasePath, m.AnnouncementsGETHandler)
}

74
internal/api/client/announcements/announcementsget.go Normal file
View File

@ -0,0 +1,74 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package announcements
import (
"net/http"
"github.com/gin-gonic/gin"
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/oauth"
)
// AnnouncementsGETHandler swagger:operation GET /api/v1/announcements announcementsGet
//
// Get an array of currently active announcements.
//
// THIS ENDPOINT IS CURRENTLY NOT FULLY IMPLEMENTED: it will always return an empty array.
//
// ---
// tags:
// - announcements
//
// produces:
// - application/json
//
// security:
// - OAuth2 Bearer:
// - read:announcements
//
// responses:
// '200':
// schema:
// type: array
// items:
// type: object
// maxItems: 0
// '400':
// description: bad request
// '401':
// description: unauthorized
// '406':
// description: not acceptable
// '500':
// description: internal server error
func (m *Module) AnnouncementsGETHandler(c *gin.Context) {
_, err := oauth.Authed(c, true, true, true, true)
if err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorUnauthorized(err, err.Error()), m.processor.InstanceGetV1)
return
}
if _, err := apiutil.NegotiateAccept(c, apiutil.JSONAcceptHeaders...); err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorNotAcceptable(err, err.Error()), m.processor.InstanceGetV1)
return
}
apiutil.JSON(c, http.StatusOK, apiutil.EmptyJSONArray)
}

2
internal/api/client/exports/exports_test.go
View File

@ -231,7 +231,7 @@ type testCase struct {
"media_storage": "",
"followers_count": 2,
"following_count": 2,
"statuses_count": 8,
"statuses_count": 9,
"lists_count": 1,
"blocks_count": 0,
"mutes_count": 0

1
internal/api/client/instance/instancepatch.go
View File

@ -175,6 +175,7 @@ func validateInstanceUpdate(form *apimodel.InstanceSettingsUpdateRequest) error
form.ContactEmail == nil &&
form.ShortDescription == nil &&
form.Description == nil &&
form.CustomCSS == nil &&
form.Terms == nil &&
form.Avatar == nil &&
form.AvatarDescription == nil &&

12
internal/api/client/instance/instancepatch_test.go
View File

@ -155,7 +155,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch1() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/assets/logo.webp",
@ -296,7 +296,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch2() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/assets/logo.webp",
@ -437,7 +437,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch3() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/assets/logo.webp",
@ -629,7 +629,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch6() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/assets/logo.webp",
@ -792,7 +792,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch8() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/fileserver/01AY6P665V14JJR0AFVRT7311Y/attachment/original/`+instanceAccount.AvatarMediaAttachment.ID+`.gif",`+`
@ -974,7 +974,7 @@ func (suite *InstancePatchTestSuite) TestInstancePatch9() {
},
"stats": {
"domain_count": 2,
"status_count": 19,
"status_count": 21,
"user_count": 4
},
"thumbnail": "http://localhost:8080/assets/logo.webp",

2
internal/api/client/mutes/mutesget_test.go
View File

@ -148,7 +148,7 @@ func (suite *MutesTestSuite) TestIndefinitelyMutedAccountSerializesMuteExpiratio
// Fetch all muted accounts for the logged-in account.
// The expected body contains `"mute_expires_at":null`.
_, err = suite.getMutedAccounts(http.StatusOK, `[{"id":"01F8MH5ZK5VRH73AKHQM6Y9VNX","username":"foss_satan","acct":"foss_satan@fossbros-anonymous.io","display_name":"big gerald","locked":false,"discoverable":true,"bot":false,"created_at":"2021-09-26T10:52:36.000Z","note":"i post about like, i dunno, stuff, or whatever!!!!","url":"http://fossbros-anonymous.io/@foss_satan","avatar":"","avatar_static":"","header":"http://localhost:8080/assets/default_header.webp","header_static":"http://localhost:8080/assets/default_header.webp","header_description":"Flat gray background (default header).","followers_count":0,"following_count":0,"statuses_count":3,"last_status_at":"2021-09-11","emojis":[],"fields":[],"mute_expires_at":null}]`)
_, err = suite.getMutedAccounts(http.StatusOK, `[{"id":"01F8MH5ZK5VRH73AKHQM6Y9VNX","username":"foss_satan","acct":"foss_satan@fossbros-anonymous.io","display_name":"big gerald","locked":false,"discoverable":true,"bot":false,"created_at":"2021-09-26T10:52:36.000Z","note":"i post about like, i dunno, stuff, or whatever!!!!","url":"http://fossbros-anonymous.io/@foss_satan","avatar":"","avatar_static":"","header":"http://localhost:8080/assets/default_header.webp","header_static":"http://localhost:8080/assets/default_header.webp","header_description":"Flat gray background (default header).","followers_count":0,"following_count":0,"statuses_count":4,"last_status_at":"2024-11-01","emojis":[],"fields":[],"mute_expires_at":null}]`)
if err != nil {
suite.FailNow(err.Error())
}

4
internal/api/client/reports/reportget_test.go
View File

@ -130,8 +130,8 @@ func (suite *ReportGetTestSuite) TestGetReport1() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}

16
internal/api/client/reports/reportsget_test.go
View File

@ -156,8 +156,8 @@ func (suite *ReportsGetTestSuite) TestGetReports() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -247,8 +247,8 @@ func (suite *ReportsGetTestSuite) TestGetReports4() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -322,8 +322,8 @@ func (suite *ReportsGetTestSuite) TestGetReports6() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}
@ -381,8 +381,8 @@ func (suite *ReportsGetTestSuite) TestGetReports7() {
"header_description": "Flat gray background (default header).",
"followers_count": 0,
"following_count": 0,
"statuses_count": 3,
"last_status_at": "2021-09-11",
"statuses_count": 4,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": []
}

6
internal/api/client/search/searchget_test.go
View File

@ -916,7 +916,7 @@ func (suite *SearchGetTestSuite) TestSearchAAny() {
}
suite.Len(searchResult.Accounts, 5)
suite.Len(searchResult.Statuses, 7)
suite.Len(searchResult.Statuses, 8)
suite.Len(searchResult.Hashtags, 0)
}
@ -959,7 +959,7 @@ func (suite *SearchGetTestSuite) TestSearchAAnyFollowingOnly() {
}
suite.Len(searchResult.Accounts, 2)
suite.Len(searchResult.Statuses, 7)
suite.Len(searchResult.Statuses, 8)
suite.Len(searchResult.Hashtags, 0)
}
@ -1002,7 +1002,7 @@ func (suite *SearchGetTestSuite) TestSearchAStatuses() {
}
suite.Len(searchResult.Accounts, 0)
suite.Len(searchResult.Statuses, 7)
suite.Len(searchResult.Statuses, 8)
suite.Len(searchResult.Hashtags, 0)
}

3
internal/api/client/statuses/status.go
View File

@ -83,9 +83,10 @@ func New(processor *processing.Processor) *Module {
}
func (m *Module) Route(attachHandler func(method string, path string, f ...gin.HandlerFunc) gin.IRoutes) {
// create / get / delete status
// create / get / edit / delete status
attachHandler(http.MethodPost, BasePath, m.StatusCreatePOSTHandler)
attachHandler(http.MethodGet, BasePathWithID, m.StatusGETHandler)
attachHandler(http.MethodPut, BasePathWithID, m.StatusEditPUTHandler)
attachHandler(http.MethodDelete, BasePathWithID, m.StatusDELETEHandler)
// fave stuff

6
internal/api/client/statuses/statusboost_test.go
View File

@ -100,6 +100,7 @@ func (suite *StatusBoostTestSuite) TestPostBoost() {
"card": null,
"content": "",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": true,
"favourites_count": 0,
@ -145,6 +146,7 @@ func (suite *StatusBoostTestSuite) TestPostBoost() {
"card": null,
"content": "hello world! #welcome ! first post on the instance :rainbow: !",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [
{
"category": "reactions",
@ -280,6 +282,7 @@ func (suite *StatusBoostTestSuite) TestPostBoostOwnFollowersOnly() {
"card": null,
"content": "",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -329,6 +332,7 @@ func (suite *StatusBoostTestSuite) TestPostBoostOwnFollowersOnly() {
"card": null,
"content": "hi!",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -494,6 +498,7 @@ func (suite *StatusBoostTestSuite) TestPostBoostImplicitAccept() {
"card": null,
"content": "",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -539,6 +544,7 @@ func (suite *StatusBoostTestSuite) TestPostBoostImplicitAccept() {
"card": null,
"content": "<p>Hi <span class=\"h-card\"><a href=\"http://localhost:8080/@1happyturtle\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>1happyturtle</span></a></span>, can I reply?</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,

156
internal/api/client/statuses/statuscreate.go
View File

@ -27,11 +27,9 @@
"github.com/go-playground/form/v4"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util"
"github.com/superseriousbusiness/gotosocial/internal/config"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/oauth"
"github.com/superseriousbusiness/gotosocial/internal/util"
"github.com/superseriousbusiness/gotosocial/internal/validate"
)
// StatusCreatePOSTHandler swagger:operation POST /api/v1/statuses statusCreate
@ -272,9 +270,9 @@ func (m *Module) StatusCreatePOSTHandler(c *gin.Context) {
return
}
form, err := parseStatusCreateForm(c)
if err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorBadRequest(err, err.Error()), m.processor.InstanceGetV1)
form, errWithCode := parseStatusCreateForm(c)
if errWithCode != nil {
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1)
return
}
@ -287,11 +285,6 @@ func (m *Module) StatusCreatePOSTHandler(c *gin.Context) {
// }
// form.Status += "\n\nsent from " + user + "'s iphone\n"
if errWithCode := validateStatusCreateForm(form); errWithCode != nil {
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1)
return
}
apiStatus, errWithCode := m.processor.Status().Create(
c.Request.Context(),
authed.Account,
@ -303,7 +296,7 @@ func (m *Module) StatusCreatePOSTHandler(c *gin.Context) {
return
}
c.JSON(http.StatusOK, apiStatus)
apiutil.JSON(c, http.StatusOK, apiStatus)
}
// intPolicyFormBinding satisfies gin's binding.Binding interface.
@ -328,108 +321,69 @@ func (intPolicyFormBinding) Bind(req *http.Request, obj any) error {
return decoder.Decode(obj, req.Form)
}
func parseStatusCreateForm(c *gin.Context) (*apimodel.StatusCreateRequest, error) {
func parseStatusCreateForm(c *gin.Context) (*apimodel.StatusCreateRequest, gtserror.WithCode) {
form := new(apimodel.StatusCreateRequest)
switch ct := c.ContentType(); ct {
case binding.MIMEJSON:
// Just bind with default json binding.
if err := c.ShouldBindWith(form, binding.JSON); err != nil {
return nil, err
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
case binding.MIMEPOSTForm:
// Bind with default form binding first.
if err := c.ShouldBindWith(form, binding.FormPost); err != nil {
return nil, err
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
// Now do custom binding.
intReqForm := new(apimodel.StatusInteractionPolicyForm)
if err := c.ShouldBindWith(intReqForm, intPolicyFormBinding{}); err != nil {
return nil, err
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
form.InteractionPolicy = intReqForm.InteractionPolicy
case binding.MIMEMultipartPOSTForm:
// Bind with default form binding first.
if err := c.ShouldBindWith(form, binding.FormMultipart); err != nil {
return nil, err
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
// Now do custom binding.
intReqForm := new(apimodel.StatusInteractionPolicyForm)
if err := c.ShouldBindWith(intReqForm, intPolicyFormBinding{}); err != nil {
return nil, err
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
form.InteractionPolicy = intReqForm.InteractionPolicy
default:
err := fmt.Errorf(
"content-type %s not supported for this endpoint; supported content-types are %s, %s, %s",
ct, binding.MIMEJSON, binding.MIMEPOSTForm, binding.MIMEMultipartPOSTForm,
)
return nil, err
}
return form, nil
}
// validateStatusCreateForm checks the form for disallowed
// combinations of attachments, overlength inputs, etc.
//
// Side effect: normalizes the post's language tag.
func validateStatusCreateForm(form *apimodel.StatusCreateRequest) gtserror.WithCode {
var (
chars = len([]rune(form.Status)) + len([]rune(form.SpoilerText))
maxChars = config.GetStatusesMaxChars()
mediaFiles = len(form.MediaIDs)
maxMediaFiles = config.GetStatusesMediaMaxFiles()
hasMedia = mediaFiles != 0
hasPoll = form.Poll != nil
)
if chars == 0 && !hasMedia && !hasPoll {
// Status must contain *some* kind of content.
const text = "no status content, content warning, media, or poll provided"
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
if chars > maxChars {
text := fmt.Sprintf(
"status too long, %d characters provided (including content warning) but limit is %d",
chars, maxChars,
)
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
if mediaFiles > maxMediaFiles {
text := fmt.Sprintf(
"too many media files attached to status, %d attached but limit is %d",
mediaFiles, maxMediaFiles,
)
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
if form.Poll != nil {
if errWithCode := validateStatusPoll(form); errWithCode != nil {
return errWithCode
}
text := fmt.Sprintf("content-type %s not supported for this endpoint; supported content-types are %s, %s, %s",
ct, binding.MIMEJSON, binding.MIMEPOSTForm, binding.MIMEMultipartPOSTForm)
return nil, gtserror.NewErrorNotAcceptable(errors.New(text), text)
}
// Check not scheduled status.
if form.ScheduledAt != "" {
const text = "scheduled_at is not yet implemented"
return gtserror.NewErrorNotImplemented(errors.New(text), text)
}
// Validate + normalize
// language tag if provided.
if form.Language != "" {
lang, err := validate.Language(form.Language)
if err != nil {
return gtserror.NewErrorBadRequest(err, err.Error())
}
form.Language = lang
return nil, gtserror.NewErrorNotImplemented(errors.New(text), text)
}
// Check if the deprecated "federated" field was
@ -438,42 +392,9 @@ func validateStatusCreateForm(form *apimodel.StatusCreateRequest) gtserror.WithC
form.LocalOnly = util.Ptr(!*form.Federated) // nolint:staticcheck
}
return nil
}
// Normalize poll expiry time if a poll was given.
if form.Poll != nil && form.Poll.ExpiresInI != nil {
func validateStatusPoll(form *apimodel.StatusCreateRequest) gtserror.WithCode {
var (
maxPollOptions = config.GetStatusesPollMaxOptions()
pollOptions = len(form.Poll.Options)
maxPollOptionChars = config.GetStatusesPollOptionMaxChars()
)
if pollOptions == 0 {
const text = "poll with no options"
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
if pollOptions > maxPollOptions {
text := fmt.Sprintf(
"too many poll options provided, %d provided but limit is %d",
pollOptions, maxPollOptions,
)
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
for _, option := range form.Poll.Options {
optionChars := len([]rune(option))
if optionChars > maxPollOptionChars {
text := fmt.Sprintf(
"poll option too long, %d characters provided but limit is %d",
optionChars, maxPollOptionChars,
)
return gtserror.NewErrorBadRequest(errors.New(text), text)
}
}
// Normalize poll expiry if necessary.
if form.Poll.ExpiresInI != nil {
// If we parsed this as JSON, expires_in
// may be either a float64 or a string.
expiresIn, err := apiutil.ParseDuration(
@ -481,13 +402,10 @@ func validateStatusPoll(form *apimodel.StatusCreateRequest) gtserror.WithCode {
"expires_in",
)
if err != nil {
return gtserror.NewErrorBadRequest(err, err.Error())
}
if expiresIn != nil {
form.Poll.ExpiresIn = *expiresIn
return nil, gtserror.NewErrorBadRequest(err, err.Error())
}
form.Poll.ExpiresIn = util.PtrOrZero(expiresIn)
}
return nil
return form, nil
}

12
internal/api/client/statuses/statuscreate_test.go
View File

@ -102,6 +102,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatus() {
"card": null,
"content": "<p>this is a brand new status! <a href=\"http://localhost:8080/tags/helloworld\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>helloworld</span></a></p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -187,6 +188,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusIntPolicy() {
"card": null,
"content": "<p>this is a brand new status! <a href=\"http://localhost:8080/tags/helloworld\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>helloworld</span></a></p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -282,6 +284,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusIntPolicyJSON() {
"card": null,
"content": "<p>this is a brand new status! <a href=\"http://localhost:8080/tags/helloworld\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>helloworld</span></a></p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -407,6 +410,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusMarkdown() {
"card": null,
"content": "<h1>Title</h1><h2>Smaller title</h2><p>This is a post written in <a href=\"https://www.markdownguide.org/\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">markdown</a></p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -490,6 +494,7 @@ func (suite *StatusCreateTestSuite) TestMentionUnknownAccount() {
"card": null,
"content": "<p>hello <span class=\"h-card\"><a href=\"https://unknown-instance.com/@brand_new_person\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>brand_new_person</span></a></span></p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -567,6 +572,7 @@ func (suite *StatusCreateTestSuite) TestPostStatusWithLinksAndTags() {
"card": null,
"content": "<p><a href=\"http://localhost:8080/tags/test\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>test</span></a> alright, should be able to post <a href=\"http://localhost:8080/tags/links\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>links</span></a> with fragments in them now, let's see........<br><br><a href=\"https://docs.gotosocial.org/en/latest/user_guide/posts/#links\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">https://docs.gotosocial.org/en/latest/user_guide/posts/#links</a><br><br><a href=\"http://localhost:8080/tags/gotosocial\" class=\"mention hashtag\" rel=\"tag nofollow noreferrer noopener\" target=\"_blank\">#<span>gotosocial</span></a><br><br>(tobi remember to pull the docker image challenge)</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -650,6 +656,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusWithEmoji() {
"card": null,
"content": "<p>here is a rainbow emoji a few times! :rainbow: :rainbow: :rainbow:<br>here's an emoji that isn't in the db: :test_emoji:</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [
{
"category": "reactions",
@ -747,6 +754,7 @@ func (suite *StatusCreateTestSuite) TestReplyToLocalStatus() {
"card": null,
"content": "<p>hello <span class=\"h-card\"><a href=\"http://localhost:8080/@1happyturtle\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>1happyturtle</span></a></span> this reply should work!</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -829,6 +837,7 @@ func (suite *StatusCreateTestSuite) TestAttachNewMediaSuccess() {
"card": null,
"content": "<p>here's an image attachment</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -933,6 +942,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusWithNoncanonicalLanguageTag
"card": null,
"content": "<p>English? what's English? i speak American</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -1007,6 +1017,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusWithPollForm() {
"card": null,
"content": "<p>this is a status with a poll!</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,
@ -1103,6 +1114,7 @@ func (suite *StatusCreateTestSuite) TestPostNewStatusWithPollJSON() {
"card": null,
"content": "<p>this is a status with a poll!</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": false,
"favourites_count": 0,

2
internal/api/client/statuses/statusdelete.go
View File

@ -95,5 +95,5 @@ func (m *Module) StatusDELETEHandler(c *gin.Context) {
return
}
c.JSON(http.StatusOK, apiStatus)
apiutil.JSON(c, http.StatusOK, apiStatus)
}

249
internal/api/client/statuses/statusedit.go Normal file
View File

@ -0,0 +1,249 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package statuses
import (
"errors"
"fmt"
"net/http"
"github.com/gin-gonic/gin"
"github.com/gin-gonic/gin/binding"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
apiutil "github.com/superseriousbusiness/gotosocial/internal/api/util"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/oauth"
"github.com/superseriousbusiness/gotosocial/internal/util"
)
// StatusEditPUTHandler swagger:operation PUT /api/v1/statuses statusEdit
//
// Edit an existing status using the given form field parameters.
//
// The parameters can also be given in the body of the request, as JSON, if the content-type is set to 'application/json'.
//
// ---
// tags:
// - statuses
//
// consumes:
// - application/json
// - application/x-www-form-urlencoded
//
// parameters:
// -
// name: status
// x-go-name: Status
// description: |-
// Text content of the status.
// If media_ids is provided, this becomes optional.
// Attaching a poll is optional while status is provided.
// type: string
// in: formData
// -
// name: media_ids
// x-go-name: MediaIDs
// description: |-
// Array of Attachment ids to be attached as media.
// If provided, status becomes optional, and poll cannot be used.
//
// If the status is being submitted as a form, the key is 'media_ids[]',
// but if it's json or xml, the key is 'media_ids'.
// type: array
// items:
// type: string
// in: formData
// -
// name: poll[options][]
// x-go-name: PollOptions
// description: |-
// Array of possible poll answers.
// If provided, media_ids cannot be used, and poll[expires_in] must be provided.
// type: array
// items:
// type: string
// in: formData
// -
// name: poll[expires_in]
// x-go-name: PollExpiresIn
// description: |-
// Duration the poll should be open, in seconds.
// If provided, media_ids cannot be used, and poll[options] must be provided.
// type: integer
// format: int64
// in: formData
// -
// name: poll[multiple]
// x-go-name: PollMultiple
// description: Allow multiple choices on this poll.
// type: boolean
// default: false
// in: formData
// -
// name: poll[hide_totals]
// x-go-name: PollHideTotals
// description: Hide vote counts until the poll ends.
// type: boolean
// default: true
// in: formData
// -
// name: sensitive
// x-go-name: Sensitive
// description: Status and attached media should be marked as sensitive.
// type: boolean
// in: formData
// -
// name: spoiler_text
// x-go-name: SpoilerText
// description: |-
// Text to be shown as a warning or subject before the actual content.
// Statuses are generally collapsed behind this field.
// type: string
// in: formData
// -
// name: language
// x-go-name: Language
// description: ISO 639 language code for this status.
// type: string
// in: formData
// -
// name: content_type
// x-go-name: ContentType
// description: Content type to use when parsing this status.
// type: string
// enum:
// - text/plain
// - text/markdown
// in: formData
//
// produces:
// - application/json
//
// security:
// - OAuth2 Bearer:
// - write:statuses
//
// responses:
// '200':
// description: "The latest status revision."
// schema:
// "$ref": "#/definitions/status"
// '400':
// description: bad request
// '401':
// description: unauthorized
// '403':
// description: forbidden
// '404':
// description: not found
// '406':
// description: not acceptable
// '500':
// description: internal server error
func (m *Module) StatusEditPUTHandler(c *gin.Context) {
authed, err := oauth.Authed(c, true, true, true, true)
if err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorUnauthorized(err, err.Error()), m.processor.InstanceGetV1)
return
}
if authed.Account.IsMoving() {
apiutil.ForbiddenAfterMove(c)
return
}
if _, err := apiutil.NegotiateAccept(c, apiutil.JSONAcceptHeaders...); err != nil {
apiutil.ErrorHandler(c, gtserror.NewErrorNotAcceptable(err, err.Error()), m.processor.InstanceGetV1)
return
}
form, errWithCode := parseStatusEditForm(c)
if errWithCode != nil {
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1)
return
}
apiStatus, errWithCode := m.processor.Status().Edit(
c.Request.Context(),
authed.Account,
c.Param(IDKey),
form,
)
if errWithCode != nil {
apiutil.ErrorHandler(c, errWithCode, m.processor.InstanceGetV1)
return
}
apiutil.JSON(c, http.StatusOK, apiStatus)
}
func parseStatusEditForm(c *gin.Context) (*apimodel.StatusEditRequest, gtserror.WithCode) {
form := new(apimodel.StatusEditRequest)
switch ct := c.ContentType(); ct {
case binding.MIMEJSON:
// Just bind with default json binding.
if err := c.ShouldBindWith(form, binding.JSON); err != nil {
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
case binding.MIMEPOSTForm:
// Bind with default form binding first.
if err := c.ShouldBindWith(form, binding.FormPost); err != nil {
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
case binding.MIMEMultipartPOSTForm:
// Bind with default form binding first.
if err := c.ShouldBindWith(form, binding.FormMultipart); err != nil {
return nil, gtserror.NewErrorBadRequest(
err,
err.Error(),
)
}
default:
text := fmt.Sprintf("content-type %s not supported for this endpoint; supported content-types are %s, %s, %s",
ct, binding.MIMEJSON, binding.MIMEPOSTForm, binding.MIMEMultipartPOSTForm)
return nil, gtserror.NewErrorNotAcceptable(errors.New(text), text)
}
// Normalize poll expiry time if a poll was given.
if form.Poll != nil && form.Poll.ExpiresInI != nil {
// If we parsed this as JSON, expires_in
// may be either a float64 or a string.
expiresIn, err := apiutil.ParseDuration(
form.Poll.ExpiresInI,
"expires_in",
)
if err != nil {
return nil, gtserror.NewErrorBadRequest(err, err.Error())
}
form.Poll.ExpiresIn = util.PtrOrZero(expiresIn)
}
return form, nil
}

32
internal/api/client/statuses/statusedit_test.go Normal file
View File

@ -0,0 +1,32 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package statuses_test
import (
"testing"
"github.com/stretchr/testify/suite"
)
type StatusEditTestSuite struct {
StatusStandardTestSuite
}
func TestStatusEditTestSuite(t *testing.T) {
suite.Run(t, new(StatusEditTestSuite))
}

2
internal/api/client/statuses/statusfave_test.go
View File

@ -105,6 +105,7 @@ func (suite *StatusFaveTestSuite) TestPostFave() {
"card": null,
"content": "🐕🐕🐕🐕🐕",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": true,
"favourites_count": 1,
@ -228,6 +229,7 @@ func (suite *StatusFaveTestSuite) TestPostFaveImplicitAccept() {
"card": null,
"content": "<p>Hi <span class=\"h-card\"><a href=\"http://localhost:8080/@1happyturtle\" class=\"u-url mention\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">@<span>1happyturtle</span></a></span>, can I reply?</p>",
"created_at": "right the hell just now babyee",
"edited_at": null,
"emojis": [],
"favourited": true,
"favourites_count": 1,

4
internal/api/client/statuses/statushistory_test.go
View File

@ -116,8 +116,8 @@ func (suite *StatusHistoryTestSuite) TestGetHistory() {
"header_media_id": "01PFPMWK2FF0D9WMHEJHR07C3Q",
"followers_count": 2,
"following_count": 2,
"statuses_count": 8,
"last_status_at": "2024-01-10",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [],
"enable_rss": true

10
internal/api/client/statuses/statusmute_test.go
View File

@ -91,6 +91,7 @@ func (suite *StatusMuteTestSuite) TestMuteUnmuteStatus() {
suite.Equal(`{
"id": "01F8MHAMCHF6Y650WCRSCP4WMY",
"created_at": "2021-10-20T10:40:37.000Z",
"edited_at": null,
"in_reply_to_id": null,
"in_reply_to_account_id": null,
"sensitive": true,
@ -134,8 +135,8 @@ func (suite *StatusMuteTestSuite) TestMuteUnmuteStatus() {
"header_media_id": "01PFPMWK2FF0D9WMHEJHR07C3Q",
"followers_count": 2,
"following_count": 2,
"statuses_count": 8,
"last_status_at": "2024-01-10",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [],
"enable_rss": true
@ -178,6 +179,7 @@ func (suite *StatusMuteTestSuite) TestMuteUnmuteStatus() {
suite.Equal(`{
"id": "01F8MHAMCHF6Y650WCRSCP4WMY",
"created_at": "2021-10-20T10:40:37.000Z",
"edited_at": null,
"in_reply_to_id": null,
"in_reply_to_account_id": null,
"sensitive": true,
@ -221,8 +223,8 @@ func (suite *StatusMuteTestSuite) TestMuteUnmuteStatus() {
"header_media_id": "01PFPMWK2FF0D9WMHEJHR07C3Q",
"followers_count": 2,
"following_count": 2,
"statuses_count": 8,
"last_status_at": "2024-01-10",
"statuses_count": 9,
"last_status_at": "2024-11-01",
"emojis": [],
"fields": [],
"enable_rss": true

2
internal/api/client/statuses/statussource_test.go
View File

@ -91,7 +91,7 @@ func (suite *StatusSourceTestSuite) TestGetSource() {
suite.Equal(`{
"id": "01F8MHAMCHF6Y650WCRSCP4WMY",
"text": "**STATUS EDITS ARE NOT CURRENTLY SUPPORTED IN GOTOSOCIAL (coming in 2024)**\nYou can review the original text of your status below, but you will not be able to submit this edit.\n\n---\n\nhello everyone!",
"text": "hello everyone!",
"spoiler_text": "introduction post"
}`, dst.String())
}

25
internal/api/model/attachment.go
View File

@ -23,12 +23,15 @@
//
// swagger: ignore
type AttachmentRequest struct {
// Media file.
File *multipart.FileHeader `form:"file" binding:"required"`
// Description of the media file. Optional.
// This will be used as alt-text for users of screenreaders etc.
// example: This is an image of some kittens, they are very cute and fluffy.
Description string `form:"description"`
// Focus of the media file. Optional.
// If present, it should be in the form of two comma-separated floats between -1 and 1.
// example: -0.5,0.565
@ -39,16 +42,38 @@ type AttachmentRequest struct {
//
// swagger:ignore
type AttachmentUpdateRequest struct {
// Description of the media file.
// This will be used as alt-text for users of screenreaders etc.
// allowEmptyValue: true
Description *string `form:"description" json:"description" xml:"description"`
// Focus of the media file.
// If present, it should be in the form of two comma-separated floats between -1 and 1.
// allowEmptyValue: true
Focus *string `form:"focus" json:"focus" xml:"focus"`
}
// AttachmentAttributesRequest models an edit request for attachment attributes.
//
// swagger:ignore
type AttachmentAttributesRequest struct {
// The ID of the attachment.
// example: 01FC31DZT1AYWDZ8XTCRWRBYRK
ID string `form:"id" json:"id"`
// Description of the media file.
// This will be used as alt-text for users of screenreaders etc.
// allowEmptyValue: true
Description string `form:"description" json:"description"`
// Focus of the media file.
// If present, it should be in the form of two comma-separated floats between -1 and 1.
// allowEmptyValue: true
Focus string `form:"focus" json:"focus"`
}
// Attachment models a media attachment.
//
// swagger:model attachment

3
internal/api/model/content.go
View File

@ -19,7 +19,6 @@
import (
"io"
"time"
"github.com/superseriousbusiness/gotosocial/internal/storage"
)
@ -30,8 +29,6 @@ type Content struct {
ContentType string
// ContentLength in bytes
ContentLength int64
// Time when the content was last updated.
ContentUpdated time.Time
// Actual content
Content io.ReadCloser
// Resource URL to forward to if the file can be fetched from the storage directly (e.g signed S3 URL)

2
internal/api/model/instance.go
View File

@ -33,6 +33,8 @@ type InstanceSettingsUpdateRequest struct {
ShortDescription *string `form:"short_description" json:"short_description" xml:"short_description"`
// Longer description of the instance, max 5,000 chars. HTML formatting accepted.
Description *string `form:"description" json:"description" xml:"description"`
// Custom CSS for the instance.
CustomCSS *string `form:"custom_css" json:"custom_css,omitempty" xml:"custom_css"`
// Terms and conditions of the instance, max 5,000 chars. HTML formatting accepted.
Terms *string `form:"terms" json:"terms" xml:"terms"`
// Image to use as the instance thumbnail.

2
internal/api/model/instancev1.go
View File

@ -38,6 +38,8 @@ type InstanceV1 struct {
//
// This should be displayed on the 'about' page for an instance.
Description string `json:"description"`
// Custom CSS for the instance.
CustomCSS string `json:"custom_css,omitempty"`
// Raw (unparsed) version of description.
DescriptionText string `json:"description_text,omitempty"`
// A shorter description of the instance.

2
internal/api/model/instancev2.go
View File

@ -53,6 +53,8 @@ type InstanceV2 struct {
Description string `json:"description"`
// Raw (unparsed) version of description.
DescriptionText string `json:"description_text,omitempty"`
// Instance Custom Css
CustomCSS string `json:"custom_css,omitempty"`
// Basic anonymous usage data for this instance.
Usage InstanceV2Usage `json:"usage"`
// An image used to represent this instance.

74
internal/api/model/status.go
View File

@ -29,6 +29,10 @@ type Status struct {
// The date when this status was created (ISO 8601 Datetime).
// example: 2021-07-30T09:20:25+00:00
CreatedAt string `json:"created_at"`
// Timestamp of when the status was last edited (ISO 8601 Datetime).
// example: 2021-07-30T09:20:25+00:00
// nullable: true
EditedAt *string `json:"edited_at"`
// ID of the status being replied to.
// example: 01FBVD42CQ3ZEEVMW180SBX03B
// nullable: true
@ -193,36 +197,50 @@ type StatusReblogged struct {
//
// swagger:ignore
type StatusCreateRequest struct {
// Text content of the status.
// If media_ids is provided, this becomes optional.
// Attaching a poll is optional while status is provided.
Status string `form:"status" json:"status"`
// Array of Attachment ids to be attached as media.
// If provided, status becomes optional, and poll cannot be used.
MediaIDs []string `form:"media_ids[]" json:"media_ids"`
// Poll to include with this status.
Poll *PollRequest `form:"poll" json:"poll"`
// ID of the status being replied to, if status is a reply.
InReplyToID string `form:"in_reply_to_id" json:"in_reply_to_id"`
// Status and attached media should be marked as sensitive.
Sensitive bool `form:"sensitive" json:"sensitive"`
// Text to be shown as a warning or subject before the actual content.
// Statuses are generally collapsed behind this field.
SpoilerText string `form:"spoiler_text" json:"spoiler_text"`
// Visibility of the posted status.
Visibility Visibility `form:"visibility" json:"visibility"`
// Set to "true" if this status should not be federated, ie. it should be a "local only" status.
// Set to "true" if this status should not be
// federated,ie. it should be a "local only" status.
LocalOnly *bool `form:"local_only" json:"local_only"`
// Deprecated: Only used if LocalOnly is not set.
Federated *bool `form:"federated" json:"federated"`
// ISO 8601 Datetime at which to schedule a status.
// Providing this parameter will cause ScheduledStatus to be returned instead of Status.
// Must be at least 5 minutes in the future.
ScheduledAt string `form:"scheduled_at" json:"scheduled_at"`
// ISO 639 language code for this status.
Language string `form:"language" json:"language"`
// Content type to use when parsing this status.
ContentType StatusContentType `form:"content_type" json:"content_type"`
// Interaction policy to use for this status.
InteractionPolicy *InteractionPolicy `form:"-" json:"interaction_policy"`
}
@ -232,6 +250,7 @@ type StatusCreateRequest struct {
//
// swagger:ignore
type StatusInteractionPolicyForm struct {
// Interaction policy to use for this status.
InteractionPolicy *InteractionPolicy `form:"interaction_policy" json:"-"`
}
@ -246,13 +265,18 @@ type StatusInteractionPolicyForm struct {
// VisibilityNone is visible to nobody. This is only used for the visibility of web statuses.
VisibilityNone Visibility = "none"
// VisibilityPublic is visible to everyone, and will be available via the web even for nonauthenticated users.
VisibilityPublic Visibility = "public"
// VisibilityUnlisted is visible to everyone, but only on home timelines, lists, etc.
VisibilityUnlisted Visibility = "unlisted"
// VisibilityPrivate is visible only to followers of the account that posted the status.
VisibilityPrivate Visibility = "private"
// VisibilityMutualsOnly is visible only to mutual followers of the account that posted the status.
VisibilityMutualsOnly Visibility = "mutuals_only"
// VisibilityDirect is visible only to accounts tagged in the status. It is equivalent to a direct message.
VisibilityDirect Visibility = "direct"
)
@ -264,7 +288,8 @@ type StatusInteractionPolicyForm struct {
// swagger:type string
type StatusContentType string
// Content type to use when parsing submitted status into an html-formatted status
// Content type to use when parsing submitted
// status into an html-formatted status.
const (
StatusContentTypePlain StatusContentType = "text/plain"
StatusContentTypeMarkdown StatusContentType = "text/markdown"
@ -276,11 +301,14 @@ type StatusInteractionPolicyForm struct {
//
// swagger:model statusSource
type StatusSource struct {
// ID of the status.
// example: 01FBVD42CQ3ZEEVMW180SBX03B
ID string `json:"id"`
// Plain-text source of a status.
Text string `json:"text"`
// Plain-text version of spoiler text.
SpoilerText string `json:"spoiler_text"`
}
@ -290,27 +318,69 @@ type StatusSource struct {
//
// swagger:model statusEdit
type StatusEdit struct {
// The content of this status at this revision.
// Should be HTML, but might also be plaintext in some cases.
// example: <p>Hey this is a status!</p>
Content string `json:"content"`
// Subject, summary, or content warning for the status at this revision.
// example: warning nsfw
SpoilerText string `json:"spoiler_text"`
// Status marked sensitive at this revision.
// example: false
Sensitive bool `json:"sensitive"`
// The date when this revision was created (ISO 8601 Datetime).
// example: 2021-07-30T09:20:25+00:00
CreatedAt string `json:"created_at"`
// The account that authored this status.
Account *Account `json:"account"`
// The poll attached to the status at this revision.
// Note that edits changing the poll options will be collapsed together into one edit, since this action resets the poll.
// nullable: true
Poll *Poll `json:"poll"`
// Media that is attached to this status.
MediaAttachments []*Attachment `json:"media_attachments"`
// Custom emoji to be used when rendering status content.
Emojis []Emoji `json:"emojis"`
}
// StatusEditRequest models status edit parameters.
//
// swagger:ignore
type StatusEditRequest struct {
// Text content of the status.
// If media_ids is provided, this becomes optional.
// Attaching a poll is optional while status is provided.
Status string `form:"status" json:"status"`
// Text to be shown as a warning or subject before the actual content.
// Statuses are generally collapsed behind this field.
SpoilerText string `form:"spoiler_text" json:"spoiler_text"`
// Content type to use when parsing this status.
ContentType StatusContentType `form:"content_type" json:"content_type"`
// Status and attached media should be marked as sensitive.
Sensitive bool `form:"sensitive" json:"sensitive"`
// ISO 639 language code for this status.
Language string `form:"language" json:"language"`
// Array of Attachment ids to be attached as media.
// If provided, status becomes optional, and poll cannot be used.
MediaIDs []string `form:"media_ids[]" json:"media_ids"`
// Array of Attachment attributes to be updated in attached media.
MediaAttributes []AttachmentAttributesRequest `form:"media_attributes[]" json:"media_attributes"`
// Poll to include with this status.
Poll *PollRequest `form:"poll" json:"poll"`
}

42
internal/api/util/parseform.go
View File

@ -18,13 +18,55 @@
package util
import (
"errors"
"fmt"
"strconv"
"strings"
apimodel "github.com/superseriousbusiness/gotosocial/internal/api/model"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/util"
)
// ParseFocus parses a media attachment focus parameters from incoming API string.
func ParseFocus(focus string) (focusx, focusy float32, errWithCode gtserror.WithCode) {
if focus == "" {
return
}
spl := strings.Split(focus, ",")
if len(spl) != 2 {
const text = "missing comma separator"
errWithCode = gtserror.NewErrorBadRequest(
errors.New(text),
text,
)
return
}
xStr := spl[0]
yStr := spl[1]
fx, err := strconv.ParseFloat(xStr, 32)
if err != nil || fx > 1 || fx < -1 {
text := fmt.Sprintf("invalid x focus: %s", xStr)
errWithCode = gtserror.NewErrorBadRequest(
errors.New(text),
text,
)
return
}
fy, err := strconv.ParseFloat(yStr, 32)
if err != nil || fy > 1 || fy < -1 {
text := fmt.Sprintf("invalid y focus: %s", xStr)
errWithCode = gtserror.NewErrorBadRequest(
errors.New(text),
text,
)
return
}
focusx = float32(fx)
focusy = float32(fy)
return
}
// ParseDuration parses the given raw interface belonging
// the given fieldName as an integer duration.
func ParseDuration(rawI any, fieldName string) (*int, error) {

1
internal/cache/cache.go vendored
View File

@ -105,6 +105,7 @@ func (c *Caches) Init() {
c.initStatus()
c.initStatusBookmark()
c.initStatusBookmarkIDs()
c.initStatusEdit()
c.initStatusFave()
c.initStatusFaveIDs()
c.initTag()

35
internal/cache/db.go vendored
View File

@ -226,6 +226,9 @@ type DBCaches struct {
// StatusBookmarkIDs provides access to the status bookmark IDs list database cache.
StatusBookmarkIDs SliceCache[string]
// StatusEdit provides access to the gtsmodel StatusEdit database cache.
StatusEdit StructCache[*gtsmodel.StatusEdit]
// StatusFave provides access to the gtsmodel StatusFave database cache.
StatusFave StructCache[*gtsmodel.StatusFave]
@ -1394,6 +1397,38 @@ func (c *Caches) initStatusBookmarkIDs() {
c.DB.StatusBookmarkIDs.Init(0, cap)
}
func (c *Caches) initStatusEdit() {
// Calculate maximum cache size.
cap := calculateResultCacheMax(
sizeofStatusEdit(), // model in-mem size.
config.GetCacheStatusEditMemRatio(),
)
log.Infof(nil, "cache size = %d", cap)
copyF := func(s1 *gtsmodel.StatusEdit) *gtsmodel.StatusEdit {
s2 := new(gtsmodel.StatusEdit)
*s2 = *s1
// Don't include ptr fields that
// will be populated separately.
s2.Attachments = nil
return s2
}
c.DB.StatusEdit.Init(structr.CacheConfig[*gtsmodel.StatusEdit]{
Indices: []structr.IndexConfig{
{Fields: "ID"},
{Fields: "StatusID", Multiple: true},
},
MaxSize: cap,
IgnoreErr: ignoreErrors,
Copy: copyF,
Invalidate: c.OnInvalidateStatusEdit,
})
}
func (c *Caches) initStatusFave() {
// Calculate maximum cache size.
cap := calculateResultCacheMax(

5
internal/cache/invalidate.go vendored
View File

@ -273,6 +273,11 @@ func (c *Caches) OnInvalidateStatusBookmark(bookmark *gtsmodel.StatusBookmark) {
c.DB.StatusBookmarkIDs.Invalidate(bookmark.StatusID)
}
func (c *Caches) OnInvalidateStatusEdit(edit *gtsmodel.StatusEdit) {
// Invalidate cache of related status model.
c.DB.Status.Invalidate("ID", edit.StatusID)
}
func (c *Caches) OnInvalidateStatusFave(fave *gtsmodel.StatusFave) {
// Invalidate status fave ID list for this status.
c.DB.StatusFaveIDs.Invalidate(fave.StatusID)

19
internal/cache/size.go vendored
View File

@ -513,7 +513,6 @@ func sizeofMedia() uintptr {
URL: exampleURI,
RemoteURL: exampleURI,
CreatedAt: exampleTime,
UpdatedAt: exampleTime,
Type: gtsmodel.FileTypeImage,
AccountID: exampleID,
Description: exampleText,
@ -540,7 +539,6 @@ func sizeofMention() uintptr {
ID: exampleURI,
StatusID: exampleURI,
CreatedAt: exampleTime,
UpdatedAt: exampleTime,
OriginAccountID: exampleURI,
OriginAccountURI: exampleURI,
TargetAccountID: exampleID,
@ -682,6 +680,23 @@ func sizeofStatusBookmark() uintptr {
}))
}
func sizeofStatusEdit() uintptr {
return uintptr(size.Of(&gtsmodel.StatusEdit{
ID: exampleID,
Content: exampleText,
ContentWarning: exampleUsername, // similar length
Text: exampleText,
Language: "en",
Sensitive: func() *bool { ok := false; return &ok }(),
AttachmentIDs: []string{exampleID, exampleID, exampleID},
Attachments: nil,
PollOptions: []string{exampleTextSmall, exampleTextSmall, exampleTextSmall, exampleTextSmall},
PollVotes: []int{69, 420, 1337, 1969},
StatusID: exampleID,
CreatedAt: exampleTime,
}))
}
func sizeofStatusFave() uintptr {
return uintptr(size.Of(&gtsmodel.StatusFave{
ID: exampleID,

1
internal/config/config.go
View File

@ -238,6 +238,7 @@ type CacheConfiguration struct {
StatusMemRatio float64 `name:"status-mem-ratio"`
StatusBookmarkMemRatio float64 `name:"status-bookmark-mem-ratio"`
StatusBookmarkIDsMemRatio float64 `name:"status-bookmark-ids-mem-ratio"`
StatusEditMemRatio float64 `name:"status-edit-mem-ratio"`
StatusFaveMemRatio float64 `name:"status-fave-mem-ratio"`
StatusFaveIDsMemRatio float64 `name:"status-fave-ids-mem-ratio"`
TagMemRatio float64 `name:"tag-mem-ratio"`

1
internal/config/defaults.go
View File

@ -199,6 +199,7 @@
StatusMemRatio: 5,
StatusBookmarkMemRatio: 0.5,
StatusBookmarkIDsMemRatio: 2,
StatusEditMemRatio: 2,
StatusFaveMemRatio: 2,
StatusFaveIDsMemRatio: 3,
TagMemRatio: 2,

25
internal/config/helpers.gen.go
View File

@ -3912,6 +3912,31 @@ func GetCacheStatusBookmarkIDsMemRatio() float64 { return global.GetCacheStatusB
// SetCacheStatusBookmarkIDsMemRatio safely sets the value for global configuration 'Cache.StatusBookmarkIDsMemRatio' field
func SetCacheStatusBookmarkIDsMemRatio(v float64) { global.SetCacheStatusBookmarkIDsMemRatio(v) }
// GetCacheStatusEditMemRatio safely fetches the Configuration value for state's 'Cache.StatusEditMemRatio' field
func (st *ConfigState) GetCacheStatusEditMemRatio() (v float64) {
st.mutex.RLock()
v = st.config.Cache.StatusEditMemRatio
st.mutex.RUnlock()
return
}
// SetCacheStatusEditMemRatio safely sets the Configuration value for state's 'Cache.StatusEditMemRatio' field
func (st *ConfigState) SetCacheStatusEditMemRatio(v float64) {
st.mutex.Lock()
defer st.mutex.Unlock()
st.config.Cache.StatusEditMemRatio = v
st.reloadToViper()
}
// CacheStatusEditMemRatioFlag returns the flag name for the 'Cache.StatusEditMemRatio' field
func CacheStatusEditMemRatioFlag() string { return "cache-status-edit-mem-ratio" }
// GetCacheStatusEditMemRatio safely fetches the value for global configuration 'Cache.StatusEditMemRatio' field
func GetCacheStatusEditMemRatio() float64 { return global.GetCacheStatusEditMemRatio() }
// SetCacheStatusEditMemRatio safely sets the value for global configuration 'Cache.StatusEditMemRatio' field
func SetCacheStatusEditMemRatio(v float64) { global.SetCacheStatusEditMemRatio(v) }
// GetCacheStatusFaveMemRatio safely fetches the Configuration value for state's 'Cache.StatusFaveMemRatio' field
func (st *ConfigState) GetCacheStatusFaveMemRatio() (v float64) {
st.mutex.RLock()

10
internal/db/bundb/account_test.go
View File

@ -46,7 +46,7 @@ type AccountTestSuite struct {
func (suite *AccountTestSuite) TestGetAccountStatuses() {
statuses, err := suite.db.GetAccountStatuses(context.Background(), suite.testAccounts["local_account_1"].ID, 20, false, false, "", "", false, false)
suite.NoError(err)
suite.Len(statuses, 8)
suite.Len(statuses, 9)
}
func (suite *AccountTestSuite) TestGetAccountStatusesPageDown() {
@ -69,7 +69,7 @@ func (suite *AccountTestSuite) TestGetAccountStatusesPageDown() {
if err != nil {
suite.FailNow(err.Error())
}
suite.Len(statuses, 2)
suite.Len(statuses, 3)
// try to get the last page (should be empty)
statuses, err = suite.db.GetAccountStatuses(context.Background(), suite.testAccounts["local_account_1"].ID, 3, false, false, statuses[len(statuses)-1].ID, "", false, false)
@ -80,13 +80,13 @@ func (suite *AccountTestSuite) TestGetAccountStatusesPageDown() {
func (suite *AccountTestSuite) TestGetAccountStatusesExcludeRepliesAndReblogs() {
statuses, err := suite.db.GetAccountStatuses(context.Background(), suite.testAccounts["local_account_1"].ID, 20, true, true, "", "", false, false)
suite.NoError(err)
suite.Len(statuses, 7)
suite.Len(statuses, 8)
}
func (suite *AccountTestSuite) TestGetAccountStatusesExcludeRepliesAndReblogsPublicOnly() {
statuses, err := suite.db.GetAccountStatuses(context.Background(), suite.testAccounts["local_account_1"].ID, 20, true, true, "", "", false, true)
suite.NoError(err)
suite.Len(statuses, 3)
suite.Len(statuses, 4)
}
// populateTestStatus adds mandatory fields to a partially populated status.
@ -173,7 +173,7 @@ func (suite *AccountTestSuite) TestGetAccountStatusesExcludeRepliesExcludesSelfR
testAccount := suite.testAccounts["local_account_1"]
statuses, err := suite.db.GetAccountStatuses(context.Background(), testAccount.ID, 20, true, true, "", "", false, false)
suite.NoError(err)
suite.Len(statuses, 8)
suite.Len(statuses, 9)
for _, status := range statuses {
if status.InReplyToID != "" && status.InReplyToAccountID != testAccount.ID {
suite.FailNowf("", "Status with ID %s is a non-self reply and should have been excluded", status.ID)

2
internal/db/bundb/basic_test.go
View File

@ -114,7 +114,7 @@ func (suite *BasicTestSuite) TestGetAllStatuses() {
s := []*gtsmodel.Status{}
err := suite.db.GetAll(context.Background(), &s)
suite.NoError(err)
suite.Len(s, 25)
suite.Len(s, 28)
}
func (suite *BasicTestSuite) TestGetAllNotNull() {

5
internal/db/bundb/bundb.go
View File

@ -81,6 +81,7 @@ type DBService struct {
db.SinBinStatus
db.Status
db.StatusBookmark
db.StatusEdit
db.StatusFave
db.Tag
db.Thread
@ -273,6 +274,10 @@ func NewBunDBService(ctx context.Context, state *state.State) (db.DB, error) {
db: db,
state: state,
},
StatusEdit: &statusEditDB{
db: db,
state: state,
},
StatusFave: &statusFaveDB{
db: db,
state: state,

2
internal/db/bundb/bundb_test.go
View File

@ -57,6 +57,7 @@ type BunDBStandardTestSuite struct {
testPolls map[string]*gtsmodel.Poll
testPollVotes map[string]*gtsmodel.PollVote
testInteractionRequests map[string]*gtsmodel.InteractionRequest
testStatusEdits map[string]*gtsmodel.StatusEdit
}
func (suite *BunDBStandardTestSuite) SetupSuite() {
@ -83,6 +84,7 @@ func (suite *BunDBStandardTestSuite) SetupSuite() {
suite.testPolls = testrig.NewTestPolls()
suite.testPollVotes = testrig.NewTestPollVotes()
suite.testInteractionRequests = testrig.NewTestInteractionRequests()
suite.testStatusEdits = testrig.NewTestStatusEdits()
}
func (suite *BunDBStandardTestSuite) SetupTest() {

4
internal/db/bundb/instance_test.go
View File

@ -47,13 +47,13 @@ func (suite *InstanceTestSuite) TestCountInstanceUsersRemote() {
func (suite *InstanceTestSuite) TestCountInstanceStatuses() {
count, err := suite.db.CountInstanceStatuses(context.Background(), config.GetHost())
suite.NoError(err)
suite.Equal(19, count)
suite.Equal(21, count)
}
func (suite *InstanceTestSuite) TestCountInstanceStatusesRemote() {
count, err := suite.db.CountInstanceStatuses(context.Background(), "fossbros-anonymous.io")
suite.NoError(err)
suite.Equal(3, count)
suite.Equal(4, count)
}
func (suite *InstanceTestSuite) TestCountInstanceDomains() {

18
internal/db/bundb/interaction_test.go
View File

@ -59,11 +59,7 @@ func (suite *InteractionTestSuite) markInteractionsPending(
// Put an interaction request
// in the DB for this reply.
req, err := typeutils.StatusToInteractionRequest(ctx, reply)
if err != nil {
suite.FailNow(err.Error())
}
req := typeutils.StatusToInteractionRequest(reply)
if err := suite.state.DB.PutInteractionRequest(ctx, req); err != nil {
suite.FailNow(err.Error())
}
@ -90,11 +86,7 @@ func (suite *InteractionTestSuite) markInteractionsPending(
// Put an interaction request
// in the DB for this boost.
req, err := typeutils.StatusToInteractionRequest(ctx, boost)
if err != nil {
suite.FailNow(err.Error())
}
req := typeutils.StatusToInteractionRequest(boost)
if err := suite.state.DB.PutInteractionRequest(ctx, req); err != nil {
suite.FailNow(err.Error())
}
@ -121,11 +113,7 @@ func (suite *InteractionTestSuite) markInteractionsPending(
// Put an interaction request
// in the DB for this fave.
req, err := typeutils.StatusFaveToInteractionRequest(ctx, fave)
if err != nil {
suite.FailNow(err.Error())
}
req := typeutils.StatusFaveToInteractionRequest(fave)
if err := suite.state.DB.PutInteractionRequest(ctx, req); err != nil {
suite.FailNow(err.Error())
}

6
internal/db/bundb/media.go
View File

@ -104,12 +104,6 @@ func (m *mediaDB) PutAttachment(ctx context.Context, media *gtsmodel.MediaAttach
}
func (m *mediaDB) UpdateAttachment(ctx context.Context, media *gtsmodel.MediaAttachment, columns ...string) error {
media.UpdatedAt = time.Now()
if len(columns) > 0 {
// If we're updating by column, ensure "updated_at" is included.
columns = append(columns, "updated_at")
}
return m.state.Caches.DB.Media.Store(media, func() error {
_, err := m.db.NewUpdate().
Model(media).

11
internal/db/bundb/migrations/20240809134448_interaction_requests_client_api.go
View File

@ -93,11 +93,7 @@ func init() {
// For each currently pending status, check whether it's a reply or
// a boost, and insert a corresponding interaction request into the db.
for _, pendingStatus := range pendingStatuses {
req, err := typeutils.StatusToInteractionRequest(ctx, pendingStatus)
if err != nil {
return err
}
req := typeutils.StatusToInteractionRequest(pendingStatus)
if _, err := tx.
NewInsert().
Model(req).
@ -125,10 +121,7 @@ func init() {
}
for _, pendingFave := range pendingFaves {
req, err := typeutils.StatusFaveToInteractionRequest(ctx, pendingFave)
if err != nil {
return err
}
req := typeutils.StatusFaveToInteractionRequest(pendingFave)
if _, err := tx.
NewInsert().

44
internal/db/bundb/migrations/20240924222938_add_instance_custom_css.go Normal file
View File

@ -0,0 +1,44 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package migrations
import (
"context"
"strings"
"github.com/uptrace/bun"
)
func init() {
up := func(ctx context.Context, db *bun.DB) error {
_, err := db.ExecContext(ctx, "ALTER TABLE ? ADD COLUMN ? TEXT", bun.Ident("instances"), bun.Ident("custom_css"))
if err != nil && !(strings.Contains(err.Error(), "already exists") || strings.Contains(err.Error(), "duplicate column name") || strings.Contains(err.Error(), "SQLSTATE 42701")) {
return err
}
return nil
}
down := func(ctx context.Context, db *bun.DB) error {
_, err := db.ExecContext(ctx, "ALTER TABLE ? DROP COLUMN ?", bun.Ident("instances"), bun.Ident("custom_css"))
return err
}
if err := Migrations.Register(up, down); err != nil {
panic(err)
}
}

59
internal/db/bundb/migrations/20241113151042_remove_mention_updated_at.go Normal file
View File

@ -0,0 +1,59 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package migrations
import (
"context"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun"
)
func init() {
up := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
// Check for 'updated_at' column on mentions table, else return.
exists, err := doesColumnExist(ctx, tx, "mentions", "updated_at")
if err != nil {
return err
} else if !exists {
return nil
}
// Remove 'updated_at' column.
log.Info(ctx, "removing unused updated_at column from mentions to save space, please wait...")
_, err = tx.NewDropColumn().
Model((*gtsmodel.Mention)(nil)).
Column("updated_at").
Exec(ctx)
return err
})
}
down := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
return nil
})
}
if err := Migrations.Register(up, down); err != nil {
panic(err)
}
}

69
internal/db/bundb/migrations/20241113152126_add_status_edits.go Normal file
View File

@ -0,0 +1,69 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package migrations
import (
"context"
"reflect"
gtsmodel "github.com/superseriousbusiness/gotosocial/internal/db/bundb/migrations/20241113152126_add_status_edits"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun"
)
func init() {
up := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
statusType := reflect.TypeOf((*gtsmodel.Status)(nil))
// Generate new Status.EditIDs column definition from bun.
colDef, err := getBunColumnDef(tx, statusType, "EditIDs")
if err != nil {
return err
}
// Add EditIDs column to Status table.
log.Info(ctx, "adding edits column to statuses table...")
_, err = tx.NewAddColumn().
Model((*gtsmodel.Status)(nil)).
ColumnExpr(colDef).
Exec(ctx)
if err != nil {
return err
}
// Create the main StatusEdits table.
_, err = tx.NewCreateTable().
IfNotExists().
Model((*gtsmodel.StatusEdit)(nil)).
Exec(ctx)
return err
})
}
down := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
return nil
})
}
if err := Migrations.Register(up, down); err != nil {
panic(err)
}
}

97
internal/db/bundb/migrations/20241113152126_add_status_edits/status.go Normal file
View File

@ -0,0 +1,97 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package gtsmodel
import (
"time"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
)
// Status represents a user-created 'post' or 'status' in the database, either remote or local
type Status struct {
ID string `bun:"type:CHAR(26),pk,nullzero,notnull,unique"` // id of this item in the database
CreatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item created
UpdatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item last updated
FetchedAt time.Time `bun:"type:timestamptz,nullzero"` // when was item (remote) last fetched.
PinnedAt time.Time `bun:"type:timestamptz,nullzero"` // Status was pinned by owning account at this time.
URI string `bun:",unique,nullzero,notnull"` // activitypub URI of this status
URL string `bun:",nullzero"` // web url for viewing this status
Content string `bun:""` // content of this status; likely html-formatted but not guaranteed
AttachmentIDs []string `bun:"attachments,array"` // Database IDs of any media attachments associated with this status
Attachments []*gtsmodel.MediaAttachment `bun:"attached_media,rel:has-many"` // Attachments corresponding to attachmentIDs
TagIDs []string `bun:"tags,array"` // Database IDs of any tags used in this status
Tags []*gtsmodel.Tag `bun:"attached_tags,m2m:status_to_tags"` // Tags corresponding to tagIDs. https://bun.uptrace.dev/guide/relations.html#many-to-many-relation
MentionIDs []string `bun:"mentions,array"` // Database IDs of any mentions in this status
Mentions []*gtsmodel.Mention `bun:"attached_mentions,rel:has-many"` // Mentions corresponding to mentionIDs
EmojiIDs []string `bun:"emojis,array"` // Database IDs of any emojis used in this status
Emojis []*gtsmodel.Emoji `bun:"attached_emojis,m2m:status_to_emojis"` // Emojis corresponding to emojiIDs. https://bun.uptrace.dev/guide/relations.html#many-to-many-relation
Local *bool `bun:",nullzero,notnull,default:false"` // is this status from a local account?
AccountID string `bun:"type:CHAR(26),nullzero,notnull"` // which account posted this status?
Account *gtsmodel.Account `bun:"rel:belongs-to"` // account corresponding to accountID
AccountURI string `bun:",nullzero,notnull"` // activitypub uri of the owner of this status
InReplyToID string `bun:"type:CHAR(26),nullzero"` // id of the status this status replies to
InReplyToURI string `bun:",nullzero"` // activitypub uri of the status this status is a reply to
InReplyToAccountID string `bun:"type:CHAR(26),nullzero"` // id of the account that this status replies to
InReplyTo *Status `bun:"-"` // status corresponding to inReplyToID
InReplyToAccount *gtsmodel.Account `bun:"rel:belongs-to"` // account corresponding to inReplyToAccountID
BoostOfID string `bun:"type:CHAR(26),nullzero"` // id of the status this status is a boost of
BoostOfURI string `bun:"-"` // URI of the status this status is a boost of; field not inserted in the db, just for dereferencing purposes.
BoostOfAccountID string `bun:"type:CHAR(26),nullzero"` // id of the account that owns the boosted status
BoostOf *Status `bun:"-"` // status that corresponds to boostOfID
BoostOfAccount *gtsmodel.Account `bun:"rel:belongs-to"` // account that corresponds to boostOfAccountID
ThreadID string `bun:"type:CHAR(26),nullzero"` // id of the thread to which this status belongs; only set for remote statuses if a local account is involved at some point in the thread, otherwise null
EditIDs []string `bun:"edits,array"` //
Edits []*StatusEdit `bun:"-"` //
PollID string `bun:"type:CHAR(26),nullzero"` //
Poll *gtsmodel.Poll `bun:"-"` //
ContentWarning string `bun:",nullzero"` // cw string for this status
Visibility Visibility `bun:",nullzero,notnull"` // visibility entry for this status
Sensitive *bool `bun:",nullzero,notnull,default:false"` // mark the status as sensitive?
Language string `bun:",nullzero"` // what language is this status written in?
CreatedWithApplicationID string `bun:"type:CHAR(26),nullzero"` // Which application was used to create this status?
CreatedWithApplication *gtsmodel.Application `bun:"rel:belongs-to"` // application corresponding to createdWithApplicationID
ActivityStreamsType string `bun:",nullzero,notnull"` // What is the activitystreams type of this status? See: https://www.w3.org/TR/activitystreams-vocabulary/#object-types. Will probably almost always be Note but who knows!.
Text string `bun:""` // Original text of the status without formatting
Federated *bool `bun:",notnull"` // This status will be federated beyond the local timeline(s)
InteractionPolicy *gtsmodel.InteractionPolicy `bun:""` // InteractionPolicy for this status. If null then the default InteractionPolicy should be assumed for this status's Visibility. Always null for boost wrappers.
PendingApproval *bool `bun:",nullzero,notnull,default:false"` // If true then status is a reply or boost wrapper that must be Approved by the reply-ee or boost-ee before being fully distributed.
PreApproved bool `bun:"-"` // If true, then status is a reply to or boost wrapper of a status on our instance, has permission to do the interaction, and an Accept should be sent out for it immediately. Field not stored in the DB.
ApprovedByURI string `bun:",nullzero"` // URI of an Accept Activity that approves the Announce or Create Activity that this status was/will be attached to.
}
// Visibility represents the visibility granularity of a status.
type Visibility string
const (
// VisibilityNone means nobody can see this.
// It's only used for web status visibility.
VisibilityNone Visibility = "none"
// VisibilityPublic means this status will be visible to everyone on all timelines.
VisibilityPublic Visibility = "public"
// VisibilityUnlocked means this status will be visible to everyone, but will only show on home timeline to followers, and in lists.
VisibilityUnlocked Visibility = "unlocked"
// VisibilityFollowersOnly means this status is viewable to followers only.
VisibilityFollowersOnly Visibility = "followers_only"
// VisibilityMutualsOnly means this status is visible to mutual followers only.
VisibilityMutualsOnly Visibility = "mutuals_only"
// VisibilityDirect means this status is visible only to mentioned recipients.
VisibilityDirect Visibility = "direct"
// VisibilityDefault is used when no other setting can be found.
VisibilityDefault Visibility = VisibilityUnlocked
)

48
internal/db/bundb/migrations/20241113152126_add_status_edits/statusedit.go Normal file
View File

@ -0,0 +1,48 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package gtsmodel
import (
"time"
)
// StatusEdit represents a **historical** view of a Status
// after a received edit. The Status itself will always
// contain the latest up-to-date information.
//
// Note that stored status edits may not exactly match that
// of the origin server, they are a best-effort by receiver
// to store version history. There is no AP history endpoint.
type StatusEdit struct {
ID string `bun:"type:CHAR(26),pk,nullzero,notnull,unique"` // ID of this item in the database.
Content string `bun:""` // Content of status at time of edit; likely html-formatted but not guaranteed.
ContentWarning string `bun:",nullzero"` // Content warning of status at time of edit.
Text string `bun:""` // Original status text, without formatting, at time of edit.
Language string `bun:",nullzero"` // Status language at time of edit.
Sensitive *bool `bun:",nullzero,notnull,default:false"` // Status sensitive flag at time of edit.
AttachmentIDs []string `bun:"attachments,array"` // Database IDs of media attachments associated with status at time of edit.
AttachmentDescriptions []string `bun:",array"` // Previous media descriptions of media attachments associated with status at time of edit.
PollOptions []string `bun:",array"` // Poll options of status at time of edit, only set if status contains a poll.
PollVotes []int `bun:",array"` // Poll vote count at time of status edit, only set if poll votes were reset.
StatusID string `bun:"type:CHAR(26),nullzero,notnull"` // The originating status ID this is a historical edit of.
CreatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // The creation time of this version of the status content (according to receiving server).
// We don't bother having a *gtsmodel.Status model here
// as the StatusEdit is always just attached to a Status,
// so it doesn't need a self-reference back to it.
}

97
internal/db/bundb/migrations/20241121121623_enum_strings_to_ints.go
View File

@ -19,10 +19,8 @@
import (
"context"
"errors"
old_gtsmodel "github.com/superseriousbusiness/gotosocial/internal/db/bundb/migrations/20241121121623_enum_strings_to_ints"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
new_gtsmodel "github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/util"
@ -71,7 +69,9 @@ func init() {
// Before making changes to the visibility col
// we must drop all indices that rely on it.
log.Info(ctx, "dropping old visibility indexes...")
for _, index := range visIndices {
log.Info(ctx, "dropping old index %s...", index.name)
if _, err := tx.NewDropIndex().
Index(index.name).
Exec(ctx); err != nil {
@ -91,7 +91,9 @@ func init() {
}
// Recreate the visibility indices.
log.Info(ctx, "creating new visibility indexes...")
for _, index := range visIndices {
log.Info(ctx, "creating new index %s...", index.name)
q := tx.NewCreateIndex().
Table("statuses").
Index(index.name).
@ -128,97 +130,6 @@ func init() {
}
}
// convertEnums performs a transaction that converts
// a table's column of our old-style enums (strings) to
// more performant and space-saving integer types.
func convertEnums[OldType ~string, NewType ~int16](
ctx context.Context,
tx bun.Tx,
table string,
column string,
mapping map[OldType]NewType,
defaultValue *NewType,
) error {
if len(mapping) == 0 {
return errors.New("empty mapping")
}
// Generate new column name.
newColumn := column + "_new"
log.Infof(ctx, "converting %s.%s enums; "+
"this may take a while, please don't interrupt!",
table, column,
)
// Ensure a default value.
if defaultValue == nil {
var zero NewType
defaultValue = &zero
}
// Add new column to database.
if _, err := tx.NewAddColumn().
Table(table).
ColumnExpr("? SMALLINT NOT NULL DEFAULT ?",
bun.Ident(newColumn),
*defaultValue).
Exec(ctx); err != nil {
return gtserror.Newf("error adding new column: %w", err)
}
// Get a count of all in table.
total, err := tx.NewSelect().
Table(table).
Count(ctx)
if err != nil {
return gtserror.Newf("error selecting total count: %w", err)
}
var updated int
for old, new := range mapping {
// Update old to new values.
res, err := tx.NewUpdate().
Table(table).
Where("? = ?", bun.Ident(column), old).
Set("? = ?", bun.Ident(newColumn), new).
Exec(ctx)
if err != nil {
return gtserror.Newf("error updating old column values: %w", err)
}
// Count number items updated.
n, _ := res.RowsAffected()
updated += int(n)
}
// Check total updated.
if total != updated {
log.Warnf(ctx, "total=%d does not match updated=%d", total, updated)
}
// Drop the old column from table.
if _, err := tx.NewDropColumn().
Table(table).
ColumnExpr("?", bun.Ident(column)).
Exec(ctx); err != nil {
return gtserror.Newf("error dropping old column: %w", err)
}
// Rename new to old name.
if _, err := tx.NewRaw(
"ALTER TABLE ? RENAME COLUMN ? TO ?",
bun.Ident(table),
bun.Ident(newColumn),
bun.Ident(column),
).Exec(ctx); err != nil {
return gtserror.Newf("error renaming new column: %w", err)
}
return nil
}
// visibilityEnumMapping maps old Visibility enum values to their newer integer type.
func visibilityEnumMapping[T ~string]() map[T]new_gtsmodel.Visibility {
return map[T]new_gtsmodel.Visibility{

59
internal/db/bundb/migrations/20241203124608_remove_media_attachment_updated_at.go Normal file
View File

@ -0,0 +1,59 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package migrations
import (
"context"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun"
)
func init() {
up := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
// Check for 'updated_at' column on media attachments table, else return.
exists, err := doesColumnExist(ctx, tx, "media_attachments", "updated_at")
if err != nil {
return err
} else if !exists {
return nil
}
// Remove 'updated_at' column.
log.Info(ctx, "removing unused updated_at column from media attachments to save space, please wait...")
_, err = tx.NewDropColumn().
Model((*gtsmodel.MediaAttachment)(nil)).
Column("updated_at").
Exec(ctx)
return err
})
}
down := func(ctx context.Context, db *bun.DB) error {
return db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error {
return nil
})
}
if err := Migrations.Register(up, down); err != nil {
panic(err)
}
}

198
internal/db/bundb/migrations/util.go
View File

@ -19,11 +19,209 @@
import (
"context"
"errors"
"fmt"
"reflect"
"strconv"
"strings"
"codeberg.org/gruf/go-byteutil"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/uptrace/bun"
"github.com/uptrace/bun/dialect"
"github.com/uptrace/bun/dialect/feature"
"github.com/uptrace/bun/dialect/sqltype"
"github.com/uptrace/bun/schema"
)
// convertEnums performs a transaction that converts
// a table's column of our old-style enums (strings) to
// more performant and space-saving integer types.
func convertEnums[OldType ~string, NewType ~int16](
ctx context.Context,
tx bun.Tx,
table string,
column string,
mapping map[OldType]NewType,
defaultValue *NewType,
) error {
if len(mapping) == 0 {
return errors.New("empty mapping")
}
// Generate new column name.
newColumn := column + "_new"
log.Infof(ctx, "converting %s.%s enums; "+
"this may take a while, please don't interrupt!",
table, column,
)
// Ensure a default value.
if defaultValue == nil {
var zero NewType
defaultValue = &zero
}
// Add new column to database.
if _, err := tx.NewAddColumn().
Table(table).
ColumnExpr("? SMALLINT NOT NULL DEFAULT ?",
bun.Ident(newColumn),
*defaultValue).
Exec(ctx); err != nil {
return gtserror.Newf("error adding new column: %w", err)
}
// Get a count of all in table.
total, err := tx.NewSelect().
Table(table).
Count(ctx)
if err != nil {
return gtserror.Newf("error selecting total count: %w", err)
}
var updated int
for old, new := range mapping {
// Update old to new values.
res, err := tx.NewUpdate().
Table(table).
Where("? = ?", bun.Ident(column), old).
Set("? = ?", bun.Ident(newColumn), new).
Exec(ctx)
if err != nil {
return gtserror.Newf("error updating old column values: %w", err)
}
// Count number items updated.
n, _ := res.RowsAffected()
updated += int(n)
}
// Check total updated.
if total != updated {
log.Warnf(ctx, "total=%d does not match updated=%d", total, updated)
}
// Drop the old column from table.
if _, err := tx.NewDropColumn().
Table(table).
ColumnExpr("?", bun.Ident(column)).
Exec(ctx); err != nil {
return gtserror.Newf("error dropping old column: %w", err)
}
// Rename new to old name.
if _, err := tx.NewRaw(
"ALTER TABLE ? RENAME COLUMN ? TO ?",
bun.Ident(table),
bun.Ident(newColumn),
bun.Ident(column),
).Exec(ctx); err != nil {
return gtserror.Newf("error renaming new column: %w", err)
}
return nil
}
// getBunColumnDef generates a column definition string for the SQL table represented by
// Go type, with the SQL column represented by the given Go field name. This ensures when
// adding a new column for table by migration that it will end up as bun would create it.
//
// NOTE: this function must stay in sync with (*bun.CreateTableQuery{}).AppendQuery(),
// specifically where it loops over table fields appending each column definition.
func getBunColumnDef(db bun.IDB, rtype reflect.Type, fieldName string) (string, error) {
d := db.Dialect()
f := d.Features()
// Get bun schema definitions for Go type and its field.
field, table, err := getModelField(db, rtype, fieldName)
if err != nil {
return "", err
}
// Start with reasonable buf.
buf := make([]byte, 0, 64)
// Start with the SQL column name.
buf = append(buf, field.SQLName...)
buf = append(buf, " "...)
// Append the SQL
// type information.
switch {
// Most of the time these two will match, but for the cases where DiscoveredSQLType is dialect-specific,
// e.g. pgdialect would change sqltype.SmallInt to pgTypeSmallSerial for columns that have `bun:",autoincrement"`
case !strings.EqualFold(field.CreateTableSQLType, field.DiscoveredSQLType):
buf = append(buf, field.CreateTableSQLType...)
// For all common SQL types except VARCHAR, both UserDefinedSQLType and DiscoveredSQLType specify the correct type,
// and we needn't modify it. For VARCHAR columns, we will stop to check if a valid length has been set in .Varchar(int).
case !strings.EqualFold(field.CreateTableSQLType, sqltype.VarChar):
buf = append(buf, field.CreateTableSQLType...)
// All else falls back
// to a default varchar.
default:
if d.Name() == dialect.Oracle {
buf = append(buf, "VARCHAR2"...)
} else {
buf = append(buf, sqltype.VarChar...)
}
buf = append(buf, "("...)
buf = strconv.AppendInt(buf, int64(d.DefaultVarcharLen()), 10)
buf = append(buf, ")"...)
}
// Append not null definition if field requires.
if field.NotNull && d.Name() != dialect.Oracle {
buf = append(buf, " NOT NULL"...)
}
// Append autoincrement definition if field requires.
if field.Identity && f.Has(feature.GeneratedIdentity) ||
(field.AutoIncrement && (f.Has(feature.AutoIncrement) || f.Has(feature.Identity))) {
buf = d.AppendSequence(buf, table, field)
}
// Append any default value.
if field.SQLDefault != "" {
buf = append(buf, " DEFAULT "...)
buf = append(buf, field.SQLDefault...)
}
return byteutil.B2S(buf), nil
}
// getModelField returns the uptrace/bun schema details for given Go type and field name.
func getModelField(db bun.IDB, rtype reflect.Type, fieldName string) (*schema.Field, *schema.Table, error) {
// Get the associated table for Go type.
table := db.Dialect().Tables().Get(rtype)
if table == nil {
return nil, nil, fmt.Errorf("no table found for type: %s", rtype)
}
var field *schema.Field
// Look for field matching Go name.
for i := range table.Fields {
if table.Fields[i].GoName == fieldName {
field = table.Fields[i]
break
}
}
if field == nil {
return nil, nil, fmt.Errorf("no bun field found on %s with name: %s", rtype, fieldName)
}
return field, table, nil
}
// doesColumnExist safely checks whether given column exists on table, handling both SQLite and PostgreSQL appropriately.
func doesColumnExist(ctx context.Context, tx bun.Tx, table, col string) (bool, error) {
var n int

5
internal/db/bundb/poll.go
View File

@ -88,12 +88,15 @@ func (p *pollDB) getPoll(ctx context.Context, lookup string, dbQuery func(*gtsmo
func (p *pollDB) GetOpenPolls(ctx context.Context) ([]*gtsmodel.Poll, error) {
var pollIDs []string
// Select all polls with unset `closed_at` time.
// Select all polls with:
// - UNSET `closed_at`
// - SET `expires_at`
if err := p.db.NewSelect().
Table("polls").
Column("polls.id").
Join("JOIN ? ON ? = ?", bun.Ident("statuses"), bun.Ident("polls.id"), bun.Ident("statuses.poll_id")).
Where("? = true", bun.Ident("statuses.local")).
Where("? IS NOT NULL", bun.Ident("polls.expires_at")).
Where("? IS NULL", bun.Ident("polls.closed_at")).
Scan(ctx, &pollIDs); err != nil {
return nil, err

54
internal/db/bundb/status.go
View File

@ -21,7 +21,6 @@
"context"
"errors"
"slices"
"time"
"github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtscontext"
@ -181,7 +180,7 @@ func (s *statusDB) getStatus(ctx context.Context, lookup string, dbQuery func(*g
func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status) error {
var (
err error
errs = gtserror.NewMultiError(9)
errs gtserror.MultiError
)
if status.Account == nil {
@ -257,7 +256,7 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
if !status.AttachmentsPopulated() {
// Status attachments are out-of-date with IDs, repopulate.
status.Attachments, err = s.state.DB.GetAttachmentsByIDs(
ctx, // these are already barebones
gtscontext.SetBarebones(ctx),
status.AttachmentIDs,
)
if err != nil {
@ -268,7 +267,7 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
if !status.TagsPopulated() {
// Status tags are out-of-date with IDs, repopulate.
status.Tags, err = s.state.DB.GetTags(
ctx,
gtscontext.SetBarebones(ctx),
status.TagIDs,
)
if err != nil {
@ -279,7 +278,7 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
if !status.MentionsPopulated() {
// Status mentions are out-of-date with IDs, repopulate.
status.Mentions, err = s.state.DB.GetMentions(
ctx, // leave fully populated for now
ctx, // TODO: manually populate mentions for places expecting these populated
status.MentionIDs,
)
if err != nil {
@ -290,7 +289,7 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
if !status.EmojisPopulated() {
// Status emojis are out-of-date with IDs, repopulate.
status.Emojis, err = s.state.DB.GetEmojisByIDs(
ctx, // these are already barebones
gtscontext.SetBarebones(ctx),
status.EmojiIDs,
)
if err != nil {
@ -301,7 +300,7 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
if status.CreatedWithApplicationID != "" && status.CreatedWithApplication == nil {
// Populate the status' expected CreatedWithApplication (not always set).
status.CreatedWithApplication, err = s.state.DB.GetApplicationByID(
ctx, // these are already barebones
gtscontext.SetBarebones(ctx),
status.CreatedWithApplicationID,
)
if err != nil {
@ -312,6 +311,23 @@ func (s *statusDB) PopulateStatus(ctx context.Context, status *gtsmodel.Status)
return errs.Combine()
}
func (s *statusDB) PopulateStatusEdits(ctx context.Context, status *gtsmodel.Status) error {
var err error
if !status.EditsPopulated() {
// Status edits are out-of-date with IDs, repopulate.
status.Edits, err = s.state.DB.GetStatusEditsByIDs(
gtscontext.SetBarebones(ctx),
status.EditIDs,
)
if err != nil {
return gtserror.Newf("error populating status edits: %w", err)
}
}
return nil
}
func (s *statusDB) PutStatus(ctx context.Context, status *gtsmodel.Status) error {
return s.state.Caches.DB.Status.Store(status, func() error {
// It is safe to run this database transaction within cache.Store
@ -350,14 +366,14 @@ func (s *statusDB) PutStatus(ctx context.Context, status *gtsmodel.Status) error
}
}
// change the status ID of the media attachments to the new status
// change the status ID of the media
// attachments to the current status
for _, a := range status.Attachments {
a.StatusID = status.ID
a.UpdatedAt = time.Now()
if _, err := tx.
NewUpdate().
Model(a).
Column("status_id", "updated_at").
Column("status_id").
Where("? = ?", bun.Ident("media_attachment.id"), a.ID).
Exec(ctx); err != nil {
if !errors.Is(err, db.ErrAlreadyExists) {
@ -384,19 +400,15 @@ func (s *statusDB) PutStatus(ctx context.Context, status *gtsmodel.Status) error
}
// Finally, insert the status
_, err := tx.NewInsert().Model(status).Exec(ctx)
_, err := tx.NewInsert().
Model(status).
Exec(ctx)
return err
})
})
}
func (s *statusDB) UpdateStatus(ctx context.Context, status *gtsmodel.Status, columns ...string) error {
status.UpdatedAt = time.Now()
if len(columns) > 0 {
// If we're updating by column, ensure "updated_at" is included.
columns = append(columns, "updated_at")
}
return s.state.Caches.DB.Status.Store(status, func() error {
// It is safe to run this database transaction within cache.Store
// as the cache does not attempt a mutex lock until AFTER hook.
@ -434,13 +446,14 @@ func (s *statusDB) UpdateStatus(ctx context.Context, status *gtsmodel.Status, co
}
}
// change the status ID of the media attachments to the new status
// change the status ID of the media
// attachments to the current status.
for _, a := range status.Attachments {
a.StatusID = status.ID
a.UpdatedAt = time.Now()
if _, err := tx.
NewUpdate().
Model(a).
Column("status_id").
Where("? = ?", bun.Ident("media_attachment.id"), a.ID).
Exec(ctx); err != nil {
if !errors.Is(err, db.ErrAlreadyExists) {
@ -467,8 +480,7 @@ func (s *statusDB) UpdateStatus(ctx context.Context, status *gtsmodel.Status, co
}
// Finally, update the status
_, err := tx.
NewUpdate().
_, err := tx.NewUpdate().
Model(status).
Column(columns...).
Where("? = ?", bun.Ident("status.id"), status.ID).

198
internal/db/bundb/statusedit.go Normal file
View File

@ -0,0 +1,198 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package bundb
import (
"context"
"errors"
"slices"
"github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/gtscontext"
"github.com/superseriousbusiness/gotosocial/internal/gtserror"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/state"
"github.com/superseriousbusiness/gotosocial/internal/util/xslices"
"github.com/uptrace/bun"
)
type statusEditDB struct {
db *bun.DB
state *state.State
}
func (s *statusEditDB) GetStatusEditByID(ctx context.Context, id string) (*gtsmodel.StatusEdit, error) {
// Fetch edit from database cache with loader callback.
edit, err := s.state.Caches.DB.StatusEdit.LoadOne("ID",
func() (*gtsmodel.StatusEdit, error) {
var edit gtsmodel.StatusEdit
// Not cached, load edit
// from database by its ID.
if err := s.db.NewSelect().
Model(&edit).
Where("? = ?", bun.Ident("id"), id).
Scan(ctx); err != nil {
return nil, err
}
return &edit, nil
}, id,
)
if err != nil {
return nil, err
}
if gtscontext.Barebones(ctx) {
// no need to fully populate.
return edit, nil
}
// Further populate the edit fields where applicable.
if err := s.PopulateStatusEdit(ctx, edit); err != nil {
return nil, err
}
return edit, nil
}
func (s *statusEditDB) GetStatusEditsByIDs(ctx context.Context, ids []string) ([]*gtsmodel.StatusEdit, error) {
// Load status edits for IDs via cache loader callbacks.
edits, err := s.state.Caches.DB.StatusEdit.LoadIDs("ID",
ids,
func(uncached []string) ([]*gtsmodel.StatusEdit, error) {
// Preallocate expected length of uncached edits.
edits := make([]*gtsmodel.StatusEdit, 0, len(uncached))
// Perform database query scanning
// the remaining (uncached) edit IDs.
if err := s.db.NewSelect().
Model(&edits).
Where("? IN (?)", bun.Ident("id"), bun.In(uncached)).
Scan(ctx); err != nil {
return nil, err
}
return edits, nil
},
)
if err != nil {
return nil, err
}
// Reorder the edits by their
// IDs to ensure in correct order.
getID := func(e *gtsmodel.StatusEdit) string { return e.ID }
xslices.OrderBy(edits, ids, getID)
if gtscontext.Barebones(ctx) {
// no need to fully populate.
return edits, nil
}
// Populate all loaded edits, removing those we fail to
// populate (removes needing so many nil checks everywhere).
edits = slices.DeleteFunc(edits, func(edit *gtsmodel.StatusEdit) bool {
if err := s.PopulateStatusEdit(ctx, edit); err != nil {
log.Errorf(ctx, "error populating edit %s: %v", edit.ID, err)
return true
}
return false
})
return edits, nil
}
func (s *statusEditDB) PopulateStatusEdit(ctx context.Context, edit *gtsmodel.StatusEdit) error {
var err error
var errs gtserror.MultiError
// For sub-models we only want
// barebones versions of them.
ctx = gtscontext.SetBarebones(ctx)
if !edit.AttachmentsPopulated() {
// Fetch all attachments for status edit's IDs.
edit.Attachments, err = s.state.DB.GetAttachmentsByIDs(
ctx,
edit.AttachmentIDs,
)
if err != nil {
errs.Appendf("error populating edit attachments: %w", err)
}
}
return errs.Combine()
}
func (s *statusEditDB) PutStatusEdit(ctx context.Context, edit *gtsmodel.StatusEdit) error {
return s.state.Caches.DB.StatusEdit.Store(edit, func() error {
_, err := s.db.NewInsert().Model(edit).Exec(ctx)
return err
})
}
func (s *statusEditDB) DeleteStatusEdits(ctx context.Context, ids []string) error {
// Gather necessary fields from
// deleted for cache invalidation.
deleted := make([]*gtsmodel.StatusEdit, 0, len(ids))
// Delete all edits with IDs pertaining
// to given slice, returning status IDs.
if _, err := s.db.NewDelete().
Model(&deleted).
Where("? IN (?)", bun.Ident("id"), bun.In(ids)).
Returning("?", bun.Ident("status_id")).
Exec(ctx); err != nil &&
!errors.Is(err, db.ErrNoEntries) {
return err
}
// Check for no deletes.
if len(deleted) == 0 {
return nil
}
// Invalidate all the cached status edits with IDs.
s.state.Caches.DB.StatusEdit.InvalidateIDs("ID", ids)
// With each invalidate hook mark status ID of
// edit we just called for. We only want to call
// invalidate hooks of edits from unique statuses.
invalidated := make(map[string]struct{}, 1)
// Invalidate the first delete manually, this
// opt negates need for initial hashmap lookup.
s.state.Caches.OnInvalidateStatusEdit(deleted[0])
invalidated[deleted[0].StatusID] = struct{}{}
for _, edit := range deleted {
// Check not already called for status.
_, ok := invalidated[edit.StatusID]
if ok {
continue
}
// Manually call status edit invalidate hook.
s.state.Caches.OnInvalidateStatusEdit(edit)
invalidated[edit.StatusID] = struct{}{}
}
return nil
}

168
internal/db/bundb/statusedit_test.go Normal file
View File

@ -0,0 +1,168 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package bundb_test
import (
"context"
"errors"
"reflect"
"slices"
"testing"
"time"
"github.com/stretchr/testify/suite"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
)
type StatusEditTestSuite struct {
BunDBStandardTestSuite
}
func (suite *StatusEditTestSuite) TestGetStatusEditBy() {
t := suite.T()
// Create a new context for this test.
ctx, cncl := context.WithCancel(context.Background())
defer cncl()
// Sentinel error to mark avoiding a test case.
sentinelErr := errors.New("sentinel")
for _, edit := range suite.testStatusEdits {
for lookup, dbfunc := range map[string]func() (*gtsmodel.StatusEdit, error){
"id": func() (*gtsmodel.StatusEdit, error) {
return suite.db.GetStatusEditByID(ctx, edit.ID)
},
} {
// Clear database caches.
suite.state.Caches.Init()
t.Logf("checking database lookup %q", lookup)
// Perform database function.
checkEdit, err := dbfunc()
if err != nil {
if err == sentinelErr {
continue
}
t.Errorf("error encountered for database lookup %q: %v", lookup, err)
continue
}
// Check received account data.
if !areEditsEqual(edit, checkEdit) {
t.Errorf("edit does not contain expected data: %+v", checkEdit)
continue
}
}
}
}
func (suite *StatusEditTestSuite) TestGetStatusEditsByIDs() {
t := suite.T()
// Create a new context for this test.
ctx, cncl := context.WithCancel(context.Background())
defer cncl()
// editsByStatus returns all test edits by the given status with ID.
editsByStatus := func(status *gtsmodel.Status) []*gtsmodel.StatusEdit {
var edits []*gtsmodel.StatusEdit
for _, edit := range suite.testStatusEdits {
if edit.StatusID == status.ID {
edits = append(edits, edit)
}
}
return edits
}
for _, status := range suite.testStatuses {
// Get test status edit models
// that should be found for status.
check := editsByStatus(status)
// Fetch edits for the slice of IDs attached to status from database.
edits, err := suite.state.DB.GetStatusEditsByIDs(ctx, status.EditIDs)
suite.NoError(err)
// Ensure both slices
// sorted the same.
sortEdits(check)
sortEdits(edits)
// Check whether slices of status edits match.
if !slices.EqualFunc(check, edits, areEditsEqual) {
t.Error("status edit slices do not match")
}
}
}
func (suite *StatusEditTestSuite) TestDeleteStatusEdits() {
// Create a new context for this test.
ctx, cncl := context.WithCancel(context.Background())
defer cncl()
for _, status := range suite.testStatuses {
// Delete all edits for status with given IDs from database.
err := suite.state.DB.DeleteStatusEdits(ctx, status.EditIDs)
suite.NoError(err)
// Now attempt to fetch these edits from database, should be empty.
edits, err := suite.state.DB.GetStatusEditsByIDs(ctx, status.EditIDs)
suite.NoError(err)
suite.Empty(edits)
}
}
func TestStatusEditTestSuite(t *testing.T) {
suite.Run(t, new(StatusEditTestSuite))
}
func areEditsEqual(e1, e2 *gtsmodel.StatusEdit) bool {
// Clone the 1st status edit.
e1Copy := new(gtsmodel.StatusEdit)
*e1Copy = *e1
e1 = e1Copy
// Clone the 2nd status edit.
e2Copy := new(gtsmodel.StatusEdit)
*e2Copy = *e2
e2 = e2Copy
// Clear populated sub-models.
e1.Attachments = nil
e2.Attachments = nil
// Clear database-set fields.
e1.CreatedAt = time.Time{}
e2.CreatedAt = time.Time{}
return reflect.DeepEqual(*e1, *e2)
}
func sortEdits(edits []*gtsmodel.StatusEdit) {
slices.SortFunc(edits, func(a, b *gtsmodel.StatusEdit) int {
if a.CreatedAt.Before(b.CreatedAt) {
return +1
} else if b.CreatedAt.Before(a.CreatedAt) {
return -1
}
return 0
})
}

28
internal/db/bundb/timeline.go
View File

@ -123,13 +123,8 @@ func (t *timelineDB) GetHomeTimeline(ctx context.Context, accountID string, maxI
if maxID == "" || maxID >= id.Highest {
const future = 24 * time.Hour
var err error
// don't return statuses more than 24hr in the future
maxID, err = id.NewULIDFromTime(time.Now().Add(future))
if err != nil {
return nil, err
}
maxID = id.NewULIDFromTime(time.Now().Add(future))
}
// return only statuses LOWER (ie., older) than maxID
@ -223,13 +218,8 @@ func (t *timelineDB) GetPublicTimeline(ctx context.Context, maxID string, sinceI
if maxID == "" || maxID >= id.Highest {
const future = 24 * time.Hour
var err error
// don't return statuses more than 24hr in the future
maxID, err = id.NewULIDFromTime(time.Now().Add(future))
if err != nil {
return nil, err
}
maxID = id.NewULIDFromTime(time.Now().Add(future))
}
// return only statuses LOWER (ie., older) than maxID
@ -409,13 +399,8 @@ func (t *timelineDB) GetListTimeline(
if maxID == "" || maxID >= id.Highest {
const future = 24 * time.Hour
var err error
// don't return statuses more than 24hr in the future
maxID, err = id.NewULIDFromTime(time.Now().Add(future))
if err != nil {
return nil, err
}
maxID = id.NewULIDFromTime(time.Now().Add(future))
}
// return only statuses LOWER (ie., older) than maxID
@ -508,13 +493,8 @@ func (t *timelineDB) GetTagTimeline(
if maxID == "" || maxID >= id.Highest {
const future = 24 * time.Hour
var err error
// don't return statuses more than 24hr in the future
maxID, err = id.NewULIDFromTime(time.Now().Add(future))
if err != nil {
return nil, err
}
maxID = id.NewULIDFromTime(time.Now().Add(future))
}
// return only statuses LOWER (ie., older) than maxID

21
internal/db/bundb/timeline_test.go
View File

@ -37,10 +37,7 @@ type TimelineTestSuite struct {
func getFutureStatus() *gtsmodel.Status {
theDistantFuture := time.Now().Add(876600 * time.Hour)
id, err := id.NewULIDFromTime(theDistantFuture)
if err != nil {
panic(err)
}
id := id.NewULIDFromTime(theDistantFuture)
return &gtsmodel.Status{
ID: id,
@ -182,7 +179,7 @@ func (suite *TimelineTestSuite) TestGetHomeTimelineIgnoreExclusive() {
if err != nil {
suite.FailNow(err.Error())
}
suite.checkStatuses(s, id.Highest, id.Lowest, 8)
suite.checkStatuses(s, id.Highest, id.Lowest, 9)
// Remove admin account from the exclusive list.
listEntry := suite.testListEntries["local_account_1_list_1_entry_2"]
@ -196,7 +193,7 @@ func (suite *TimelineTestSuite) TestGetHomeTimelineIgnoreExclusive() {
if err != nil {
suite.FailNow(err.Error())
}
suite.checkStatuses(s, id.Highest, id.Lowest, 12)
suite.checkStatuses(s, id.Highest, id.Lowest, 13)
}
func (suite *TimelineTestSuite) TestGetHomeTimelineNoFollowing() {
@ -228,7 +225,7 @@ func (suite *TimelineTestSuite) TestGetHomeTimelineNoFollowing() {
suite.FailNow(err.Error())
}
suite.checkStatuses(s, id.Highest, id.Lowest, 8)
suite.checkStatuses(s, id.Highest, id.Lowest, 9)
}
func (suite *TimelineTestSuite) TestGetHomeTimelineWithFutureStatus() {
@ -281,8 +278,8 @@ func (suite *TimelineTestSuite) TestGetHomeTimelineFromHighest() {
}
suite.checkStatuses(s, id.Highest, id.Lowest, 5)
suite.Equal("01J2M1HPFSS54S60Y0KYV23KJE", s[0].ID)
suite.Equal("01G36SF3V6Y6V5BF9P4R7PQG7G", s[len(s)-1].ID)
suite.Equal("01JDPZEZ77X1NX0TY9M10BK1HM", s[0].ID)
suite.Equal("01HEN2RZ8BG29Y5Z9VJC73HZW7", s[len(s)-1].ID)
}
func (suite *TimelineTestSuite) TestGetListTimelineNoParams() {
@ -296,7 +293,7 @@ func (suite *TimelineTestSuite) TestGetListTimelineNoParams() {
suite.FailNow(err.Error())
}
suite.checkStatuses(s, id.Highest, id.Lowest, 12)
suite.checkStatuses(s, id.Highest, id.Lowest, 13)
}
func (suite *TimelineTestSuite) TestGetListTimelineMaxID() {
@ -311,8 +308,8 @@ func (suite *TimelineTestSuite) TestGetListTimelineMaxID() {
}
suite.checkStatuses(s, id.Highest, id.Lowest, 5)
suite.Equal("01HEN2PRXT0TF4YDRA64FZZRN7", s[0].ID)
suite.Equal("01FF25D5Q0DH7CHD57CTRS6WK0", s[len(s)-1].ID)
suite.Equal("01JDPZEZ77X1NX0TY9M10BK1HM", s[0].ID)
suite.Equal("01FN3VJGFH10KR7S2PB0GFJZYG", s[len(s)-1].ID)
}
func (suite *TimelineTestSuite) TestGetListTimelineMinID() {

1
internal/db/db.go
View File

@ -51,6 +51,7 @@ type DB interface {
SinBinStatus
Status
StatusBookmark
StatusEdit
StatusFave
Tag
Thread

4
internal/db/status.go
View File

@ -41,8 +41,12 @@ type Status interface {
GetStatusBoost(ctx context.Context, boostOfID string, byAccountID string) (*gtsmodel.Status, error)
// PopulateStatus ensures that all sub-models of a status are populated (e.g. mentions, attachments, etc).
// Except for edits, to fetch these please call PopulateStatusEdits() .
PopulateStatus(ctx context.Context, status *gtsmodel.Status) error
// PopulateStatusEdits ensures that status' edits are fully popualted.
PopulateStatusEdits(ctx context.Context, status *gtsmodel.Status) error
// PutStatus stores one status in the database.
PutStatus(ctx context.Context, status *gtsmodel.Status) error

43
internal/db/statusedit.go Normal file
View File

@ -0,0 +1,43 @@
// GoToSocial
// Copyright (C) GoToSocial Authors admin@gotosocial.org
// SPDX-License-Identifier: AGPL-3.0-or-later
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
package db
import (
"context"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
)
type StatusEdit interface {
// GetStatusEditByID fetches the StatusEdit with given ID from the database.
GetStatusEditByID(ctx context.Context, id string) (*gtsmodel.StatusEdit, error)
// GetStatusEditsByIDs fetches all StatusEdits with given IDs from database,
// this is optimized and faster than multiple calls to GetStatusEditByID.
GetStatusEditsByIDs(ctx context.Context, ids []string) ([]*gtsmodel.StatusEdit, error)
// PopulateStatusEdit ensures the given StatusEdit's sub-models are populated.
PopulateStatusEdit(ctx context.Context, edit *gtsmodel.StatusEdit) error
// PutStatusEdit inserts the given new StatusEdit into the database.
PutStatusEdit(ctx context.Context, edit *gtsmodel.StatusEdit) error
// DeleteStatusEdits deletes the StatusEdits with given IDs from the database.
DeleteStatusEdits(ctx context.Context, ids []string) error
}

7
internal/federation/dereferencing/announce.go
View File

@ -87,7 +87,7 @@ func (d *Dereferencer) EnrichAnnounce(
boost.Federated = target.Federated
// Ensure this Announce is permitted by the Announcee.
permit, err := d.isPermittedStatus(ctx, requestUser, nil, boost)
permit, err := d.isPermittedStatus(ctx, requestUser, nil, boost, true)
if err != nil {
return nil, gtserror.Newf("error checking permitted status %s: %w", boost.URI, err)
}
@ -99,10 +99,7 @@ func (d *Dereferencer) EnrichAnnounce(
}
// Generate an ID for the boost wrapper status.
boost.ID, err = id.NewULIDFromTime(boost.CreatedAt)
if err != nil {
return nil, gtserror.Newf("error generating id: %w", err)
}
boost.ID = id.NewULIDFromTime(boost.CreatedAt)
// Store the boost wrapper status in database.
switch err = d.state.DB.PutStatus(ctx, boost); {

4
internal/federation/dereferencing/dereferencer.go
View File

@ -66,7 +66,7 @@
// causing loads of dereferencing calls.
Fresh = util.Ptr(FreshnessWindow(5 * time.Minute))
// 10 seconds.
// 5 seconds.
//
// Freshest is useful when you want an
// immediately up to date model of something
@ -74,7 +74,7 @@
//
// Be careful using this one; it can cause
// lots of unnecessary traffic if used unwisely.
Freshest = util.Ptr(FreshnessWindow(10 * time.Second))
Freshest = util.Ptr(FreshnessWindow(5 * time.Second))
)
// Dereferencer wraps logic and functionality for doing dereferencing

1
internal/federation/dereferencing/media.go
View File

@ -128,6 +128,7 @@ func (d *Dereferencer) RefreshMedia(
// Check emoji is up-to-date
// with provided extra info.
switch {
case force:
case info.Blurhash != nil &&
*info.Blurhash != attach.Blurhash:
attach.Blurhash = *info.Blurhash

639
internal/federation/dereferencing/status.go
View File

@ -35,6 +35,7 @@
"github.com/superseriousbusiness/gotosocial/internal/log"
"github.com/superseriousbusiness/gotosocial/internal/media"
"github.com/superseriousbusiness/gotosocial/internal/util"
"github.com/superseriousbusiness/gotosocial/internal/util/xslices"
)
// statusFresh returns true if the given status is still
@ -302,6 +303,7 @@ func (d *Dereferencer) enrichStatusSafely(
uri,
status,
statusable,
isNew,
)
// Check for a returned HTTP code via error.
@ -374,6 +376,7 @@ func (d *Dereferencer) enrichStatus(
uri *url.URL,
status *gtsmodel.Status,
statusable ap.Statusable,
isNew bool,
) (
*gtsmodel.Status,
ap.Statusable,
@ -476,8 +479,7 @@ func (d *Dereferencer) enrichStatus(
// Ensure the final parsed status URI or URL matches
// the input URI we fetched (or received) it as.
matches, err := util.URIMatches(
uri,
matches, err := util.URIMatches(uri,
append(
ap.GetURL(statusable), // status URL(s)
ap.GetJSONLDId(statusable), // status URI
@ -497,21 +499,18 @@ func (d *Dereferencer) enrichStatus(
)
}
var isNew bool
// Based on the original provided
// status model, determine whether
// this is a new insert / update.
if isNew = (status.ID == ""); isNew {
if isNew {
// Generate new status ID from the provided creation date.
latestStatus.ID, err = id.NewULIDFromTime(latestStatus.CreatedAt)
if err != nil {
log.Errorf(ctx, "invalid created at date (falling back to 'now'): %v", err)
latestStatus.ID = id.NewULID() // just use "now"
}
latestStatus.ID = id.NewULIDFromTime(latestStatus.CreatedAt)
} else {
// Ensure that status isn't trying to re-date itself.
if !latestStatus.CreatedAt.Equal(status.CreatedAt) {
err := gtserror.Newf("status %s 'published' changed", uri)
return nil, nil, gtserror.SetMalformed(err)
}
// Reuse existing status ID.
latestStatus.ID = status.ID
}
@ -519,7 +518,6 @@ func (d *Dereferencer) enrichStatus(
// Set latest fetch time and carry-
// over some values from "old" status.
latestStatus.FetchedAt = time.Now()
latestStatus.UpdatedAt = status.UpdatedAt
latestStatus.Local = status.Local
latestStatus.PinnedAt = status.PinnedAt
@ -538,8 +536,9 @@ func (d *Dereferencer) enrichStatus(
}
// Check if this is a permitted status we should accept.
// Function also sets "PendingApproval" bool as necessary.
permit, err := d.isPermittedStatus(ctx, requestUser, status, latestStatus)
// Function also sets "PendingApproval" bool as necessary,
// and handles removal of existing statuses no longer permitted.
permit, err := d.isPermittedStatus(ctx, requestUser, status, latestStatus, isNew)
if err != nil {
return nil, nil, gtserror.Newf("error checking permissibility for status %s: %w", uri, err)
}
@ -550,59 +549,113 @@ func (d *Dereferencer) enrichStatus(
return nil, nil, gtserror.SetNotPermitted(err)
}
// Ensure the status' mentions are populated, and pass in existing to check for changes.
if err := d.fetchStatusMentions(ctx, requestUser, status, latestStatus); err != nil {
// Insert / update any attached status poll.
pollChanged, err := d.handleStatusPoll(ctx,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error handling poll for status %s: %w", uri, err)
}
// Populate mentions associated with status, passing
// in existing status to reuse old where possible.
// (especially important here to reduce need to dereference).
mentionsChanged, err := d.fetchStatusMentions(ctx,
requestUser,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error populating mentions for status %s: %w", uri, err)
}
// Ensure the status' poll remains consistent, else reset the poll.
if err := d.fetchStatusPoll(ctx, status, latestStatus); err != nil {
return nil, nil, gtserror.Newf("error populating poll for status %s: %w", uri, err)
// Ensure status in a thread is connected.
threadChanged, err := d.threadStatus(ctx,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error handling threading for status %s: %w", uri, err)
}
// Now that we know who this status replies to (handled by ASStatusToStatus)
// and who it mentions, we can add a ThreadID to it if necessary.
if err := d.threadStatus(ctx, latestStatus); err != nil {
return nil, nil, gtserror.Newf("error checking / creating threadID for status %s: %w", uri, err)
}
// Ensure the status' tags are populated, (changes are expected / okay).
if err := d.fetchStatusTags(ctx, status, latestStatus); err != nil {
// Populate tags associated with status, passing
// in existing status to reuse old where possible.
tagsChanged, err := d.fetchStatusTags(ctx,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error populating tags for status %s: %w", uri, err)
}
// Ensure the status' media attachments are populated, passing in existing to check for changes.
if err := d.fetchStatusAttachments(ctx, requestUser, status, latestStatus); err != nil {
// Populate media attachments associated with status,
// passing in existing status to reuse old where possible
// (especially important here to reduce need to dereference).
mediaChanged, err := d.fetchStatusAttachments(ctx,
requestUser,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error populating attachments for status %s: %w", uri, err)
}
// Ensure the status' emoji attachments are populated, passing in existing to check for changes.
if err := d.fetchStatusEmojis(ctx, status, latestStatus); err != nil {
// Populate emoji associated with status, passing
// in existing status to reuse old where possible
// (especially important here to reduce need to dereference).
emojiChanged, err := d.fetchStatusEmojis(ctx,
status,
latestStatus,
)
if err != nil {
return nil, nil, gtserror.Newf("error populating emojis for status %s: %w", uri, err)
}
if isNew {
// This is new, put the status in the database.
err := d.state.DB.PutStatus(ctx, latestStatus)
if err != nil {
return nil, nil, gtserror.Newf("error putting in database: %w", err)
// Simplest case, insert this new status into the database.
if err := d.state.DB.PutStatus(ctx, latestStatus); err != nil {
return nil, nil, gtserror.Newf("error inserting new status %s: %w", uri, err)
}
} else {
// This is an existing status, update the model in the database.
if err := d.state.DB.UpdateStatus(ctx, latestStatus); err != nil {
return nil, nil, gtserror.Newf("error updating database: %w", err)
// Check for and handle any edits to status, inserting
// historical edit if necessary. Also determines status
// columns that need updating in below query.
cols, err := d.handleStatusEdit(ctx,
status,
latestStatus,
pollChanged,
mentionsChanged,
threadChanged,
tagsChanged,
mediaChanged,
emojiChanged,
)
if err != nil {
return nil, nil, gtserror.Newf("error handling edit for status %s: %w", uri, err)
}
// With returned changed columns, now update the existing status entry.
if err := d.state.DB.UpdateStatus(ctx, latestStatus, cols...); err != nil {
return nil, nil, gtserror.Newf("error updating existing status %s: %w", uri, err)
}
}
return latestStatus, statusable, nil
}
// fetchStatusMentions populates the mentions on 'status', creating
// new where needed, or using unchanged mentions from 'existing' status.
func (d *Dereferencer) fetchStatusMentions(
ctx context.Context,
requestUser string,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) error {
) (
changed bool,
err error,
) {
// Allocate new slice to take the yet-to-be created mention IDs.
status.MentionIDs = make([]string, len(status.Mentions))
@ -610,7 +663,6 @@ func (d *Dereferencer) fetchStatusMentions(
var (
mention = status.Mentions[i]
alreadyExists bool
err error
)
// Search existing status for a mention already stored,
@ -633,19 +685,16 @@ func (d *Dereferencer) fetchStatusMentions(
continue
}
// Mark status as
// having changed.
changed = true
// This mention didn't exist yet.
// Generate new ID according to status creation.
// TODO: update this to use "edited_at" when we add
// support for edited status revision history.
mention.ID, err = id.NewULIDFromTime(status.CreatedAt)
if err != nil {
log.Errorf(ctx, "invalid created at date (falling back to 'now'): %v", err)
mention.ID = id.NewULID() // just use "now"
}
// Generate new ID according to latest update.
mention.ID = id.NewULIDFromTime(status.UpdatedAt)
// Set known further mention details.
mention.CreatedAt = status.CreatedAt
mention.UpdatedAt = status.UpdatedAt
mention.CreatedAt = status.UpdatedAt
mention.OriginAccount = status.Account
mention.OriginAccountID = status.AccountID
mention.OriginAccountURI = status.AccountURI
@ -657,7 +706,7 @@ func (d *Dereferencer) fetchStatusMentions(
// Place the new mention into the database.
if err := d.state.DB.PutMention(ctx, mention); err != nil {
return gtserror.Newf("error putting mention in database: %w", err)
return changed, gtserror.Newf("error putting mention in database: %w", err)
}
// Set the *new* mention and ID.
@ -678,17 +727,42 @@ func (d *Dereferencer) fetchStatusMentions(
i++
}
return nil
return changed, nil
}
func (d *Dereferencer) threadStatus(ctx context.Context, status *gtsmodel.Status) error {
if status.InReplyTo != nil {
if parentThreadID := status.InReplyTo.ThreadID; parentThreadID != "" {
// Simplest case: parent status
// is threaded, so inherit threadID.
status.ThreadID = parentThreadID
return nil
// threadStatus ensures that given status is threaded correctly
// where necessary. that is it will inherit a thread ID from the
// existing copy if it is threaded correctly, else it will inherit
// a thread ID from a parent with existing thread, else it will
// generate a new thread ID if status mentions a local account.
func (d *Dereferencer) threadStatus(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) (
changed bool,
err error,
) {
// Check for existing status
// that is already threaded.
if existing.ThreadID != "" {
// Existing is threaded correctly.
if existing.InReplyTo == nil ||
existing.InReplyTo.ThreadID == existing.ThreadID {
status.ThreadID = existing.ThreadID
return false, nil
}
// TODO: delete incorrect thread
}
// Check for existing parent to inherit threading from.
if inReplyTo := status.InReplyTo; inReplyTo != nil &&
inReplyTo.ThreadID != "" {
status.ThreadID = inReplyTo.ThreadID
return true, nil
}
// Parent wasn't threaded. If this
@ -711,7 +785,7 @@ func(m *gtsmodel.Mention) bool {
// Status doesn't mention a
// local account, so we don't
// need to thread it.
return nil
return false, nil
}
// Status mentions a local account.
@ -719,24 +793,30 @@ func(m *gtsmodel.Mention) bool {
// it to the status.
threadID := id.NewULID()
if err := d.state.DB.PutThread(
ctx,
&gtsmodel.Thread{
ID: threadID,
},
// Insert new thread model into db.
if err := d.state.DB.PutThread(ctx,
&gtsmodel.Thread{ID: threadID},
); err != nil {
return gtserror.Newf("error inserting new thread in db: %w", err)
return false, gtserror.Newf("error inserting new thread in db: %w", err)
}
// Set thread on latest status.
status.ThreadID = threadID
return nil
return true, nil
}
// fetchStatusTags populates the tags on 'status', fetching existing
// from the database and creating new where needed. 'existing' is used
// to fetch tags that have not changed since previous stored status.
func (d *Dereferencer) fetchStatusTags(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) error {
) (
changed bool,
err error,
) {
// Allocate new slice to take the yet-to-be determined tag IDs.
status.TagIDs = make([]string, len(status.Tags))
@ -751,10 +831,14 @@ func (d *Dereferencer) fetchStatusTags(
continue
}
// Mark status as
// having changed.
changed = true
// Look for existing tag with name in the database.
existing, err := d.state.DB.GetTagByName(ctx, tag.Name)
if err != nil && !errors.Is(err, db.ErrNoEntries) {
return gtserror.Newf("db error getting tag %s: %w", tag.Name, err)
return changed, gtserror.Newf("db error getting tag %s: %w", tag.Name, err)
} else if existing != nil {
status.Tags[i] = existing
status.TagIDs[i] = existing.ID
@ -788,106 +872,21 @@ func (d *Dereferencer) fetchStatusTags(
i++
}
return nil
}
func (d *Dereferencer) fetchStatusPoll(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) error {
var (
// insertStatusPoll generates ID and inserts the poll attached to status into the database.
insertStatusPoll = func(ctx context.Context, status *gtsmodel.Status) error {
var err error
// Generate new ID for poll from the status CreatedAt.
// TODO: update this to use "edited_at" when we add
// support for edited status revision history.
status.Poll.ID, err = id.NewULIDFromTime(status.CreatedAt)
if err != nil {
log.Errorf(ctx, "invalid created at date (falling back to 'now'): %v", err)
status.Poll.ID = id.NewULID() // just use "now"
}
// Update the status<->poll links.
status.PollID = status.Poll.ID
status.Poll.StatusID = status.ID
status.Poll.Status = status
// Insert this latest poll into the database.
err = d.state.DB.PutPoll(ctx, status.Poll)
if err != nil {
return gtserror.Newf("error putting in database: %w", err)
}
return nil
}
// deleteStatusPoll deletes the poll with ID, and all attached votes, from the database.
deleteStatusPoll = func(ctx context.Context, pollID string) error {
if err := d.state.DB.DeletePollByID(ctx, pollID); err != nil {
return gtserror.Newf("error deleting existing poll from database: %w", err)
}
return nil
}
)
switch {
case existing.Poll == nil && status.Poll == nil:
// no poll before or after, nothing to do.
return nil
case existing.Poll == nil && status.Poll != nil:
// no previous poll, insert new poll!
return insertStatusPoll(ctx, status)
case status.Poll == nil:
// existing poll has been deleted, remove this.
return deleteStatusPoll(ctx, existing.PollID)
case pollChanged(existing.Poll, status.Poll):
// poll has changed since original, delete and reinsert new.
if err := deleteStatusPoll(ctx, existing.PollID); err != nil {
return err
}
return insertStatusPoll(ctx, status)
case pollUpdated(existing.Poll, status.Poll):
// Since we last saw it, the poll has updated!
// Whether that be stats, or close time.
poll := existing.Poll
poll.Closing = pollJustClosed(existing.Poll, status.Poll)
poll.ClosedAt = status.Poll.ClosedAt
poll.Voters = status.Poll.Voters
poll.Votes = status.Poll.Votes
// Update poll model in the database (specifically only the possible changed columns).
if err := d.state.DB.UpdatePoll(ctx, poll, "closed_at", "voters", "votes"); err != nil {
return gtserror.Newf("error updating poll: %w", err)
}
// Update poll on status.
status.PollID = poll.ID
status.Poll = poll
return nil
default:
// latest and existing
// polls are up to date.
poll := existing.Poll
status.PollID = poll.ID
status.Poll = poll
return nil
}
return changed, nil
}
// fetchStatusAttachments populates the attachments on 'status', creating new database
// entries where needed and dereferencing it, or using unchanged from 'existing' status.
func (d *Dereferencer) fetchStatusAttachments(
ctx context.Context,
requestUser string,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) error {
) (
changed bool,
err error,
) {
// Allocate new slice to take the yet-to-be fetched attachment IDs.
status.AttachmentIDs = make([]string, len(status.Attachments))
@ -897,9 +896,26 @@ func (d *Dereferencer) fetchStatusAttachments(
// Look for existing media attachment with remote URL first.
existing, ok := existing.GetAttachmentByRemoteURL(placeholder.RemoteURL)
if ok && existing.ID != "" {
var info media.AdditionalMediaInfo
// Ensure the existing media attachment is up-to-date and cached.
existing, err := d.updateAttachment(ctx, requestUser, existing, placeholder)
// Look for any difference in stored media description.
diff := (existing.Description != placeholder.Description)
if diff {
info.Description = &placeholder.Description
}
// If description changed,
// we mark media as changed.
changed = changed || diff
// Store any attachment updates and
// ensure media is locally cached.
existing, err := d.RefreshMedia(ctx,
requestUser,
existing,
info,
diff,
)
if err != nil {
log.Errorf(ctx, "error updating existing attachment: %v", err)
@ -915,9 +931,12 @@ func (d *Dereferencer) fetchStatusAttachments(
continue
}
// Mark status as
// having changed.
changed = true
// Load this new media attachment.
attachment, err := d.GetMedia(
ctx,
attachment, err := d.GetMedia(ctx,
requestUser,
status.AccountID,
placeholder.RemoteURL,
@ -955,42 +974,316 @@ func (d *Dereferencer) fetchStatusAttachments(
i++
}
return nil
return changed, nil
}
// fetchStatusEmojis populates the emojis on 'status', creating new database entries
// where needed and dereferencing it, or using unchanged from 'existing' status.
func (d *Dereferencer) fetchStatusEmojis(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) error {
) (
changed bool,
err error,
) {
// Fetch the updated emojis for our status.
emojis, changed, err := d.fetchEmojis(ctx,
existing.Emojis,
status.Emojis,
)
if err != nil {
return gtserror.Newf("error fetching emojis: %w", err)
return changed, gtserror.Newf("error fetching emojis: %w", err)
}
if !changed {
// Use existing status emoji objects.
status.EmojiIDs = existing.EmojiIDs
status.Emojis = existing.Emojis
return nil
return false, nil
}
// Set latest emojis.
status.Emojis = emojis
// Iterate over and set changed emoji IDs.
// Extract IDs from latest slice of emojis.
status.EmojiIDs = make([]string, len(emojis))
for i, emoji := range emojis {
status.EmojiIDs[i] = emoji.ID
}
// Combine both old and new emojis, as statuses.emojis
// keeps track of emojis for both old and current edits.
status.EmojiIDs = append(status.EmojiIDs, existing.EmojiIDs...)
status.Emojis = append(status.Emojis, existing.Emojis...)
status.EmojiIDs = xslices.Deduplicate(status.EmojiIDs)
status.Emojis = xslices.DeduplicateFunc(status.Emojis,
func(e *gtsmodel.Emoji) string { return e.ID },
)
return true, nil
}
// handleStatusPoll handles both inserting of new status poll or the
// update of an existing poll. this handles the case of simple vote
// count updates (without being classified as a change of the poll
// itself), as well as full poll changes that delete existing instance.
func (d *Dereferencer) handleStatusPoll(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
) (
changed bool,
err error,
) {
switch {
case existing.Poll == nil && status.Poll == nil:
// no poll before or after, nothing to do.
return false, nil
case existing.Poll == nil && status.Poll != nil:
// no previous poll, insert new status poll!
return true, d.insertStatusPoll(ctx, status)
case status.Poll == nil:
// existing status poll has been deleted, remove this from the database.
if err = d.state.DB.DeletePollByID(ctx, existing.Poll.ID); err != nil {
err = gtserror.Newf("error deleting poll from database: %w", err)
}
return true, err
case pollChanged(existing.Poll, status.Poll):
// existing status poll has been changed, remove this from the database.
if err = d.state.DB.DeletePollByID(ctx, existing.Poll.ID); err != nil {
return true, gtserror.Newf("error deleting poll from database: %w", err)
}
// insert latest poll version into database.
return true, d.insertStatusPoll(ctx, status)
case pollStateUpdated(existing.Poll, status.Poll):
// Since we last saw it, the poll has updated!
// Whether that be stats, or close time.
poll := existing.Poll
poll.Closing = pollJustClosed(existing.Poll, status.Poll)
poll.ClosedAt = status.Poll.ClosedAt
poll.Voters = status.Poll.Voters
poll.Votes = status.Poll.Votes
// Update poll model in the database (specifically only the possible changed columns).
if err = d.state.DB.UpdatePoll(ctx, poll, "closed_at", "voters", "votes"); err != nil {
return false, gtserror.Newf("error updating poll: %w", err)
}
// Update poll on status.
status.PollID = poll.ID
status.Poll = poll
return false, nil
default:
// latest and existing
// polls are up to date.
poll := existing.Poll
status.PollID = poll.ID
status.Poll = poll
return false, nil
}
}
// insertStatusPoll inserts an assumed new poll attached to status into the database, this
// also handles generating new ID for the poll and setting necessary fields on the status.
func (d *Dereferencer) insertStatusPoll(ctx context.Context, status *gtsmodel.Status) error {
var err error
// Generate new ID for poll from latest updated time.
status.Poll.ID = id.NewULIDFromTime(status.UpdatedAt)
// Update the status<->poll links.
status.PollID = status.Poll.ID
status.Poll.StatusID = status.ID
status.Poll.Status = status
// Insert this latest poll into the database.
err = d.state.DB.PutPoll(ctx, status.Poll)
if err != nil {
return gtserror.Newf("error putting poll in database: %w", err)
}
return nil
}
// handleStatusEdit compiles a list of changed status table columns between
// existing and latest status model, and where necessary inserts a historic
// edit of the status into the database to store its previous state. the
// returned slice is a list of columns requiring updating in the database.
func (d *Dereferencer) handleStatusEdit(
ctx context.Context,
existing *gtsmodel.Status,
status *gtsmodel.Status,
pollChanged bool,
mentionsChanged bool,
threadChanged bool,
tagsChanged bool,
mediaChanged bool,
emojiChanged bool,
) (
cols []string,
err error,
) {
var edited bool
// Preallocate max slice length.
cols = make([]string, 1, 13)
// Always update `fetched_at`.
cols[0] = "fetched_at"
// Check for edited status content.
if existing.Content != status.Content {
cols = append(cols, "content")
edited = true
}
// Check for edited status content warning.
if existing.ContentWarning != status.ContentWarning {
cols = append(cols, "content_warning")
edited = true
}
// Check for edited status sensitive flag.
if *existing.Sensitive != *status.Sensitive {
cols = append(cols, "sensitive")
edited = true
}
// Check for edited status language tag.
if existing.Language != status.Language {
cols = append(cols, "language")
edited = true
}
if pollChanged {
// Attached poll was changed.
cols = append(cols, "poll_id")
edited = true
}
if mentionsChanged {
cols = append(cols, "mentions") // i.e. MentionIDs
// Mentions changed doesn't necessarily
// indicate an edit, it may just not have
// been previously populated properly.
}
if threadChanged {
cols = append(cols, "thread_id")
// Thread changed doesn't necessarily
// indicate an edit, it may just now
// actually be included in a thread.
}
if tagsChanged {
cols = append(cols, "tags") // i.e. TagIDs
// Tags changed doesn't necessarily
// indicate an edit, it may just not have
// been previously populated properly.
}
if mediaChanged {
// Attached media was changed.
cols = append(cols, "attachments") // i.e. AttachmentIDs
edited = true
}
if emojiChanged {
// Attached emojis changed.
cols = append(cols, "emojis") // i.e. EmojiIDs
// We specifically store both *new* AND *old* edit
// revision emojis in the statuses.emojis column.
emojiByID := func(e *gtsmodel.Emoji) string { return e.ID }
status.Emojis = append(status.Emojis, existing.Emojis...)
status.Emojis = xslices.DeduplicateFunc(status.Emojis, emojiByID)
status.EmojiIDs = xslices.Gather(status.EmojiIDs[:0], status.Emojis, emojiByID)
// Emojis changed doesn't necessarily
// indicate an edit, it may just not have
// been previously populated properly.
}
if edited {
// ensure that updated_at hasn't remained the same
// but an edit was received. manually intervene here.
if status.UpdatedAt.Equal(existing.UpdatedAt) ||
status.CreatedAt.Equal(status.UpdatedAt) {
// Simply use current fetching time.
status.UpdatedAt = status.FetchedAt
}
// Status has been editted since last
// we saw it, take snapshot of existing.
var edit gtsmodel.StatusEdit
edit.ID = id.NewULIDFromTime(status.UpdatedAt)
edit.Content = existing.Content
edit.ContentWarning = existing.ContentWarning
edit.Text = existing.Text
edit.Language = existing.Language
edit.Sensitive = existing.Sensitive
edit.StatusID = status.ID
// Copy existing attachments and descriptions.
edit.AttachmentIDs = existing.AttachmentIDs
edit.Attachments = existing.Attachments
if l := len(existing.Attachments); l > 0 {
edit.AttachmentDescriptions = make([]string, l)
for i, attach := range existing.Attachments {
edit.AttachmentDescriptions[i] = attach.Description
}
}
// Edit creation is last update time.
edit.CreatedAt = existing.UpdatedAt
if existing.Poll != nil {
// Poll only set if existing contained them.
edit.PollOptions = existing.Poll.Options
if pollChanged || !*existing.Poll.HideCounts ||
!existing.Poll.ClosedAt.IsZero() {
// If the counts are allowed to be
// shown, or poll has changed, then
// include poll vote counts in edit.
edit.PollVotes = existing.Poll.Votes
}
}
// Insert this new edit of existing status into database.
if err := d.state.DB.PutStatusEdit(ctx, &edit); err != nil {
return nil, gtserror.Newf("error putting edit in database: %w", err)
}
// Add edit to list of edits on the status.
status.EditIDs = append(status.EditIDs, edit.ID)
status.Edits = append(status.Edits, &edit)
// Add edit to list of cols.
cols = append(cols, "edits")
}
if !existing.UpdatedAt.Equal(status.UpdatedAt) {
// Whether status edited or not,
// updated_at column has changed.
cols = append(cols, "updated_at")
}
return cols, nil
}
// getPopulatedMention tries to populate the given
// mention with the correct TargetAccount and (if not
// yet set) TargetAccountURI, returning the populated

10
internal/federation/dereferencing/status_permitted.go
View File

@ -62,6 +62,7 @@ func (d *Dereferencer) isPermittedStatus(
requestUser string,
existing *gtsmodel.Status,
status *gtsmodel.Status,
isNew bool,
) (
permitted bool, // is permitted?
err error,
@ -98,7 +99,7 @@ func (d *Dereferencer) isPermittedStatus(
permitted = true
}
if !permitted && existing != nil {
if !permitted && !isNew {
log.Infof(ctx, "deleting unpermitted: %s", existing.URI)
// Delete existing status from database as it's no longer permitted.
@ -110,11 +111,13 @@ func (d *Dereferencer) isPermittedStatus(
return
}
// isPermittedReply ...
func (d *Dereferencer) isPermittedReply(
ctx context.Context,
requestUser string,
reply *gtsmodel.Status,
) (bool, error) {
var (
replyURI = reply.URI // Definitely set.
inReplyToURI = reply.InReplyToURI // Definitely set.
@ -149,8 +152,7 @@ func (d *Dereferencer) isPermittedReply(
// If this status's parent was rejected,
// implicitly this reply should be too;
// there's nothing more to check here.
return false, d.unpermittedByParent(
ctx,
return false, d.unpermittedByParent(ctx,
reply,
thisReq,
parentReq,
@ -164,6 +166,7 @@ func (d *Dereferencer) isPermittedReply(
// be approved, then we should just reject it
// again, as nothing's changed since last time.
if thisRejected && acceptIRI == "" {
// Nothing changed,
// still rejected.
return false, nil
@ -174,6 +177,7 @@ func (d *Dereferencer) isPermittedReply(
// to be approved. Continue permission checks.
if inReplyTo == nil {
// If we didn't have the replied-to status
// in our database (yet), we can't check
// right now if this reply is permitted.

220
internal/federation/dereferencing/status_test.go
View File

@ -21,14 +21,21 @@
"context"
"fmt"
"testing"
"time"
"github.com/stretchr/testify/suite"
"github.com/superseriousbusiness/activity/streams"
"github.com/superseriousbusiness/gotosocial/internal/ap"
"github.com/superseriousbusiness/gotosocial/internal/db"
"github.com/superseriousbusiness/gotosocial/internal/federation/dereferencing"
"github.com/superseriousbusiness/gotosocial/internal/gtsmodel"
"github.com/superseriousbusiness/gotosocial/internal/util"
"github.com/superseriousbusiness/gotosocial/testrig"
)
// instantFreshness is the shortest possible freshness window.
var instantFreshness = util.Ptr(dereferencing.FreshnessWindow(0))
type StatusTestSuite struct {
DereferencerStandardTestSuite
}
@ -229,6 +236,219 @@ func (suite *StatusTestSuite) TestDereferenceStatusWithNonMatchingURI() {
suite.Nil(fetchedStatus)
}
func (suite *StatusTestSuite) TestDereferencerRefreshStatusUpdated() {
// Create a new context for this test.
ctx, cncl := context.WithCancel(context.Background())
defer cncl()
// The local account we will be fetching statuses as.
fetchingAccount := suite.testAccounts["local_account_1"]
// The test status in question that we will be dereferencing from "remote".
testURIStr := "https://unknown-instance.com/users/brand_new_person/statuses/01FE4NTHKWW7THT67EF10EB839"
testURI := testrig.URLMustParse(testURIStr)
testStatusable := suite.client.TestRemoteStatuses[testURIStr]
// Fetch the remote status first to load it into instance.
testStatus, statusable, err := suite.dereferencer.GetStatusByURI(ctx,
fetchingAccount.Username,
testURI,
)
suite.NotNil(statusable)
suite.NoError(err)
// Run through multiple possible edits.
for _, testCase := range []struct {
editedContent string
editedContentWarning string
editedLanguage string
editedSensitive bool
editedAttachmentIDs []string
editedPollOptions []string
editedPollVotes []int
editedAt time.Time
}{
{
editedContent: "updated status content!",
editedContentWarning: "CW: edited status content",
editedLanguage: testStatus.Language, // no change
editedSensitive: *testStatus.Sensitive, // no change
editedAttachmentIDs: testStatus.AttachmentIDs, // no change
editedPollOptions: getPollOptions(testStatus), // no change
editedPollVotes: getPollVotes(testStatus), // no change
editedAt: time.Now(),
},
} {
// Take a snapshot of current
// state of the test status.
testStatus = copyStatus(testStatus)
// Edit the "remote" statusable obj.
suite.editStatusable(testStatusable,
testCase.editedContent,
testCase.editedContentWarning,
testCase.editedLanguage,
testCase.editedSensitive,
testCase.editedAttachmentIDs,
testCase.editedPollOptions,
testCase.editedPollVotes,
testCase.editedAt,
)
// Refresh with a given statusable to updated to edited copy.
latest, statusable, err := suite.dereferencer.RefreshStatus(ctx,
fetchingAccount.Username,
testStatus,
nil, // NOTE: can provide testStatusable here to test as being received (not deref'd)
instantFreshness,
)
suite.NotNil(statusable)
suite.NoError(err)
// verify updated status details.
suite.verifyEditedStatusUpdate(
// the original status
// before any changes.
testStatus,
// latest status
// being tested.
latest,
// expected current state.
&gtsmodel.StatusEdit{
Content: testCase.editedContent,
ContentWarning: testCase.editedContentWarning,
Language: testCase.editedLanguage,
Sensitive: &testCase.editedSensitive,
AttachmentIDs: testCase.editedAttachmentIDs,
PollOptions: testCase.editedPollOptions,
PollVotes: testCase.editedPollVotes,
// createdAt never changes
},
// expected historic edit.
&gtsmodel.StatusEdit{
Content: testStatus.Content,
ContentWarning: testStatus.ContentWarning,
Language: testStatus.Language,
Sensitive: testStatus.Sensitive,
AttachmentIDs: testStatus.AttachmentIDs,
PollOptions: getPollOptions(testStatus),
PollVotes: getPollVotes(testStatus),
CreatedAt: testStatus.UpdatedAt,
},
)
}
}
// editStatusable updates the given statusable attributes.
// note that this acts on the original object, no copying.
func (suite *StatusTestSuite) editStatusable(
statusable ap.Statusable,
content string,
contentWarning string,
language string,
sensitive bool,
attachmentIDs []string, // TODO: this will require some thinking as to how ...
pollOptions []string, // TODO: this will require changing statusable type to question
pollVotes []int, // TODO: this will require changing statusable type to question
editedAt time.Time,
) {
// simply reset all mentions / emojis / tags
statusable.SetActivityStreamsTag(nil)
// Update the statusable content property + language (if set).
contentProp := streams.NewActivityStreamsContentProperty()
statusable.SetActivityStreamsContent(contentProp)
contentProp.AppendXMLSchemaString(content)
if language != "" {
contentProp.AppendRDFLangString(map[string]string{
language: content,
})
}
// Update the statusable content-warning property.
summaryProp := streams.NewActivityStreamsSummaryProperty()
statusable.SetActivityStreamsSummary(summaryProp)
summaryProp.AppendXMLSchemaString(contentWarning)
// Update the statusable sensitive property.
sensitiveProp := streams.NewActivityStreamsSensitiveProperty()
statusable.SetActivityStreamsSensitive(sensitiveProp)
sensitiveProp.AppendXMLSchemaBoolean(sensitive)
// Update the statusable updated property.
ap.SetUpdated(statusable, editedAt)
}
// verifyEditedStatusUpdate verifies that a given status has
// the expected number of historic edits, the 'current' status
// attributes (encapsulated as an edit for minimized no. args),
// and the last given 'historic' status edit attributes.
func (suite *StatusTestSuite) verifyEditedStatusUpdate(
testStatus *gtsmodel.Status, // the original model
status *gtsmodel.Status, // the status to check
current *gtsmodel.StatusEdit, // expected current state
historic *gtsmodel.StatusEdit, // historic edit we expect to have
) {
// don't use this func
// name in error msgs.
suite.T().Helper()
// Check we have expected number of edits.
previousEdits := len(testStatus.Edits)
suite.Len(status.Edits, previousEdits+1)
suite.Len(status.EditIDs, previousEdits+1)
// Check current state of status.
suite.Equal(current.Content, status.Content)
suite.Equal(current.ContentWarning, status.ContentWarning)
suite.Equal(current.Language, status.Language)
suite.Equal(*current.Sensitive, *status.Sensitive)
suite.Equal(current.AttachmentIDs, status.AttachmentIDs)
suite.Equal(current.PollOptions, getPollOptions(status))
suite.Equal(current.PollVotes, getPollVotes(status))
// Check the latest historic edit matches expected.
latestEdit := status.Edits[len(status.Edits)-1]
suite.Equal(historic.Content, latestEdit.Content)
suite.Equal(historic.ContentWarning, latestEdit.ContentWarning)
suite.Equal(historic.Language, latestEdit.Language)
suite.Equal(*historic.Sensitive, *latestEdit.Sensitive)
suite.Equal(historic.AttachmentIDs, latestEdit.AttachmentIDs)
suite.Equal(historic.PollOptions, latestEdit.PollOptions)
suite.Equal(historic.PollVotes, latestEdit.PollVotes)
suite.Equal(historic.CreatedAt, latestEdit.CreatedAt)
// The status creation date should never change.
suite.Equal(testStatus.CreatedAt, status.CreatedAt)
}
func TestStatusTestSuite(t *testing.T) {
suite.Run(t, new(StatusTestSuite))
}
// copyStatus returns a copy of the given status model (not including sub-structs).
func copyStatus(status *gtsmodel.Status) *gtsmodel.Status {
copy := new(gtsmodel.Status)
*copy = *status
return copy
}
// getPollOptions extracts poll option strings from status (if poll is set).
func getPollOptions(status *gtsmodel.Status) []string {
if status.Poll != nil {
return status.Poll.Options
}
return nil
}
// getPollVotes extracts poll vote counts from status (if poll is set).
func getPollVotes(status *gtsmodel.Status) []int {
if status.Poll != nil {
return status.Poll.Votes
}
return nil
}

6
internal/federation/dereferencing/util.go
View File

@ -52,15 +52,15 @@ func emojiChanged(existing, latest *gtsmodel.Emoji) bool {
// pollChanged returns whether a poll has changed in way that
// indicates that this should be an entirely new poll. i.e. if
// the available options have changed, or the expiry has increased.
// the available options have changed, or the expiry has changed.
func pollChanged(existing, latest *gtsmodel.Poll) bool {
return !slices.Equal(existing.Options, latest.Options) ||
!existing.ExpiresAt.Equal(latest.ExpiresAt)
}
// pollUpdated returns whether a poll has updated, i.e. if the
// pollStateUpdated returns whether a poll has updated, i.e. if
// vote counts have changed, or if it has expired / been closed.
func pollUpdated(existing, latest *gtsmodel.Poll) bool {
func pollStateUpdated(existing, latest *gtsmodel.Poll) bool {
return *existing.Voters != *latest.Voters ||
!slices.Equal(existing.Votes, latest.Votes) ||
!existing.ClosedAt.Equal(latest.ClosedAt)

2
internal/federation/federatingdb/announce_test.go
View File

@ -79,7 +79,7 @@ func (suite *AnnounceTestSuite) TestAnnounceTwice() {
// Insert the boost-of status into the
// DB cache to emulate processor handling
boost.ID, _ = id.NewULIDFromTime(boost.CreatedAt)
boost.ID = id.NewULIDFromTime(boost.CreatedAt)
suite.state.Caches.DB.Status.Put(boost)
// only the URI will be set for the boosted status

1
internal/gtsmodel/instance.go
View File

@ -34,6 +34,7 @@ type Instance struct {
ShortDescriptionText string `bun:""` // Raw text version of short description (before parsing).
Description string `bun:""` // Longer description of this instance.
DescriptionText string `bun:""` // Raw text version of long description (before parsing).
CustomCSS string `bun:",nullzero"` // Custom CSS for the instance.
Terms string `bun:""` // Terms and conditions of this instance.
TermsText string `bun:""` // Raw text version of terms (before parsing).
ContactEmail string `bun:""` // Contact email address for this instance

1
internal/gtsmodel/mediaattachment.go
View File

@ -26,7 +26,6 @@
type MediaAttachment struct {
ID string `bun:"type:CHAR(26),pk,nullzero,notnull,unique"` // id of this item in the database
CreatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item created
UpdatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item last updated
StatusID string `bun:"type:CHAR(26),nullzero"` // ID of the status to which this is attached
URL string `bun:",nullzero"` // Where can the attachment be retrieved on *this* server
RemoteURL string `bun:",nullzero"` // Where can the attachment be retrieved on a remote server (empty for local media)

1
internal/gtsmodel/mention.go
View File

@ -26,7 +26,6 @@
type Mention struct {
ID string `bun:"type:CHAR(26),pk,nullzero,notnull,unique"` // id of this item in the database
CreatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item created
UpdatedAt time.Time `bun:"type:timestamptz,nullzero,notnull,default:current_timestamp"` // when was item last updated
StatusID string `bun:"type:CHAR(26),nullzero,notnull"` // ID of the status this mention originates from
Status *Status `bun:"rel:belongs-to"` // status referred to by statusID
OriginAccountID string `bun:"type:CHAR(26),nullzero,notnull"` // ID of the mention creator account

60
internal/gtsmodel/status.go
View File

@ -20,6 +20,8 @@
import (
"slices"
"time"
"github.com/superseriousbusiness/gotosocial/internal/util/xslices"
)
// Status represents a user-created 'post' or 'status' in the database, either remote or local
@ -55,6 +57,8 @@ type Status struct {
BoostOf *Status `bun:"-"` // status that corresponds to boostOfID
BoostOfAccount *Account `bun:"rel:belongs-to"` // account that corresponds to boostOfAccountID
ThreadID string `bun:"type:CHAR(26),nullzero"` // id of the thread to which this status belongs; only set for remote statuses if a local account is involved at some point in the thread, otherwise null
EditIDs []string `bun:"edits,array"` //
Edits []*StatusEdit `bun:"-"` //
PollID string `bun:"type:CHAR(26),nullzero"` //
Poll *Poll `bun:"-"` //
ContentWarning string `bun:",nullzero"` // cw string for this status
@ -92,7 +96,8 @@ func (s *Status) GetBoostOfAccountID() string {
return s.BoostOfAccountID
}
// AttachmentsPopulated returns whether media attachments are populated according to current AttachmentIDs.
// AttachmentsPopulated returns whether media attachments
// are populated according to current AttachmentIDs.
func (s *Status) AttachmentsPopulated() bool {
if len(s.AttachmentIDs) != len(s.Attachments) {
// this is the quickest indicator.
@ -106,7 +111,8 @@ func (s *Status) AttachmentsPopulated() bool {
return true
}
// TagsPopulated returns whether tags are populated according to current TagIDs.
// TagsPopulated returns whether tags are
// populated according to current TagIDs.
func (s *Status) TagsPopulated() bool {
if len(s.TagIDs) != len(s.Tags) {
// this is the quickest indicator.
@ -120,7 +126,8 @@ func (s *Status) TagsPopulated() bool {
return true
}
// MentionsPopulated returns whether mentions are populated according to current MentionIDs.
// MentionsPopulated returns whether mentions are
// populated according to current MentionIDs.
func (s *Status) MentionsPopulated() bool {
if len(s.MentionIDs) != len(s.Mentions) {
// this is the quickest indicator.
@ -134,7 +141,8 @@ func (s *Status) MentionsPopulated() bool {
return true
}
// EmojisPopulated returns whether emojis are populated according to current EmojiIDs.
// EmojisPopulated returns whether emojis are
// populated according to current EmojiIDs.
func (s *Status) EmojisPopulated() bool {
if len(s.EmojiIDs) != len(s.Emojis) {
// this is the quickest indicator.
@ -148,6 +156,21 @@ func (s *Status) EmojisPopulated() bool {
return true
}
// EditsPopulated returns whether edits are
// populated according to current EditIDs.
func (s *Status) EditsPopulated() bool {
if len(s.EditIDs) != len(s.Edits) {
// this is quickest indicator.
return false
}
for i, id := range s.EditIDs {
if s.Edits[i].ID != id {
return false
}
}
return true
}
// EmojissUpToDate returns whether status emoji attachments of receiving status are up-to-date
// according to emoji attachments of the passed status, by comparing their emoji URIs. We don't
// use IDs as this is used to determine whether there are new emojis to fetch.
@ -247,6 +270,35 @@ func (s *Status) IsLocalOnly() bool {
return s.Federated == nil || !*s.Federated
}
// AllAttachmentIDs gathers ALL media attachment IDs from both the
// receiving Status{}, and any historical Status{}.Edits. Note that
// this function will panic if Status{}.Edits is not populated.
func (s *Status) AllAttachmentIDs() []string {
var total int
if len(s.EditIDs) != len(s.Edits) {
panic("status edits not populated")
}
// Get count of attachment IDs.
total += len(s.Attachments)
for _, edit := range s.Edits {
total += len(edit.AttachmentIDs)
}
// Start gathering of all IDs with *current* attachment IDs.
attachmentIDs := make([]string, len(s.AttachmentIDs), total)
copy(attachmentIDs, s.AttachmentIDs)
// Append IDs of historical edits.
for _, edit := range s.Edits {
attachmentIDs = append(attachmentIDs, edit.AttachmentIDs...)
}
// Deduplicate these IDs in case of shared media.
return xslices.Deduplicate(attachmentIDs)
}
// StatusToTag is an intermediate struct to facilitate the many2many relationship between a status and one or more tags.
type StatusToTag struct {
StatusID string `bun:"type:CHAR(26),unique:statustag,nullzero,notnull"`

Some files were not shown because too many files have changed in this diff Show More