Daenney 9b50151f17 [feature] Beef up our AI opt-outs (#3165) ... * [chore] Synchronise our robots.txt with upstream * [feature] Add headers to escape AI crawlers This adds 2 headers that a number of AI crawlers respect to signal that content should not be included in their datasets.		2024-08-02 18:22:39 +02:00
..
cachecontrol.go	[bugfix] Set Vary header correctly on cache-control (#1988)	2023-07-13 21:27:25 +02:00
contentsecuritypolicy_test.go	[feature] Add rate limit exceptions option, use ISO8601 for rate limit reset (#2151)	2023-08-23 14:32:27 +02:00
contentsecuritypolicy.go	[feature] Set some security related headers (#3065)	2024-07-04 10:07:02 +02:00
cors.go	[chore] Replace pinafore with semaphore (#1801)	2023-05-21 22:40:43 +02:00
extraheaders.go	[feature] Beef up our AI opt-outs (#3165)	2024-08-02 18:22:39 +02:00
gzip.go	[chore] Improve copyright header handling (#1608)	2023-03-12 16:00:57 +01:00
headerfilter_test.go	[feature] request blocking by http headers (#2409)	2023-12-18 14:18:25 +00:00
headerfilter.go	[feature] request blocking by http headers (#2409)	2023-12-18 14:18:25 +00:00
logger.go	[feature] Log pubKeyID for http-signed requests (#2501)	2024-01-09 10:41:15 +01:00
ratelimit_test.go	[feature] Add rate limit exceptions option, use ISO8601 for rate limit reset (#2151)	2023-08-23 14:32:27 +02:00
ratelimit.go	[feature] request blocking by http headers (#2409)	2023-12-18 14:18:25 +00:00
requestid.go	[chore] ensure worker contexts have request ID (#2120)	2023-08-15 17:01:01 +01:00
session_test.go	[chore] Improve copyright header handling (#1608)	2023-03-12 16:00:57 +01:00
session.go	[chore] Improve copyright header handling (#1608)	2023-03-12 16:00:57 +01:00
signaturecheck.go	[feature] Try HTTP signature validation with and without query params for incoming requests (#2591)	2024-01-31 14:15:28 +00:00
throttling_test.go	[bugfix] increases sleep time before check in throttle test, to give more leeway (#2482)	2024-01-03 10:27:55 +00:00
throttling.go	[performance] simpler throttling logic (#2407)	2023-12-16 12:53:42 +01:00
tokencheck.go	[performance] remove last of relational queries to instead rely on caches (#2091)	2023-08-10 15:08:41 +01:00
useragent.go	[feature] request blocking by http headers (#2409)	2023-12-18 14:18:25 +00:00
util.go	[feature] request blocking by http headers (#2409)	2023-12-18 14:18:25 +00:00