Commit Graph

28 Commits

Author SHA1 Message Date
Gervasio Marchand
cbc49531ad
Handle 429s gracefully (#48) 2023-12-15 19:19:39 -03:00
Gervasio Marchand
5919c3bc6f
Upgrade .net (#45) 2023-12-15 08:50:42 -03:00
Gervasio Marchand
fe7703bd9f
Handle non-json responses from instances #41 (#42) 2023-06-12 12:31:11 +00:00
Chris Wood
6dcc1a0327
Addresses issue #26 - getting tags via the Mastodon API. (#36) 2023-02-14 12:30:01 +00:00
Gervasio Marchand
fbd47ae7cd
Make the number of lines to store dynamic (#40) 2023-02-14 12:13:57 +00:00
Gervasio Marchand
d9faf231a0
Use Mastodon's API to fetch hashtagged statuses (#39) 2023-02-11 16:14:15 +00:00
Gervasio Marchand
6fd7804d6c
fix NRE when OrderedItems is null (and log it, I'm curious) (#38) 2023-02-09 12:26:25 +00:00
Gervasio Marchand
91ac0e2385
If we can't pull the robots information, don't break completely (#37) 2023-02-04 21:29:44 +00:00
Gervasio Marchand
32280c334b
Store the last 5000 urls 2023-01-12 19:16:24 -03:00
Nico Berlee
04af58dde1
Columns of PinnedTags can have Include additional tags for this column enabled
This adds not only filter capabilities, which are not relevent to this project
like `not` and `all` but also has the `any` field, which is an array where
you may add more tags.

Unfortunatly, I was unable to create a precise select query for this, as the
database field `data` is not jsonb. I think parsing json  is really the most
optimal way under these circumstances.
2022-12-25 21:28:32 +01:00
Nico Berlee
1b1a8d5d0b
Get PinnedTags from database, use them if "PinnedTags": true
This need `GRANT SELECT ON web_settings TO mastodon_read;`
Fixes #25
2022-12-24 17:09:59 +01:00
Gervasio Marchand
2c11b898ff
Moar logging (#21) 2022-12-22 12:44:34 +00:00
Gervasio Marchand
83ca6bb63b
Do up to 2 * tags concurrent requests (#20) 2022-12-22 09:37:29 -03:00
Gervasio Marchand
44b47acbfe
Improve error message when it fails to pull tags (#19) 2022-12-21 03:24:45 +00:00
Gervasio Marchand
23be734a5e
Check robots.txt and use a user agent (#17) 2022-12-21 00:15:13 -03:00
Gervasio Marchand
d535512a6b
Also build the trimmed artifacts on the required check (#18) 2022-12-21 00:12:19 -03:00
Gervasio Marchand
47a3e4acd4
Mastodon connection helper tweaks (#13) 2022-12-21 02:26:06 +00:00
Gervasio Marchand
80bae5652b
Use null reference checks (and enforce them) (#11) 2022-12-21 02:11:57 +00:00
Gervasio Marchand
518b51fcf6
Fix NRE when Tags is null and error if both tags and connection string are specified (#9) 2022-12-20 21:34:43 -03:00
Gervasio Marchand
875b2fe26e
Deprecate Sites in favor of Instances (#7) 2022-12-20 17:03:06 -03:00
Gervasio Marchand
4ccfaad15f
Make it possible to scrape followed tags from the instance (#6) 2022-12-17 22:27:04 -03:00
Gervasio Marchand
e71904e1a3
Make it trimmable 2022-12-16 01:54:49 -03:00
Gervasio Marchand
0c7589c177
catch errors pulling data from other instances 2022-12-09 18:19:15 -03:00
Gervasio Marchand
f38c7ec00d
do IsNullOrEmpty instead 2022-12-07 20:31:14 -03:00
Gervasio Marchand
082d261116
Add ability to take the fake relay api key from env variable 2022-12-07 20:28:31 -03:00
Gervasio Marchand
28fe45a075
make it more parallel 2022-12-05 08:56:11 -03:00
Gervasio Marchand
54828b9469
Rename! 2022-12-05 08:38:07 -03:00
Gervasio Marchand
4434af951e
Use FakeRelay, build a docker image and set things in a config file 2022-12-05 00:37:07 -03:00