My biggest recommendation is to just get familiar with the caching constructs that are available. I understand folks think CDNs are complicated and expensive, but they're honestly incredibly cheap and relatively easy to use.
99.9% of the time, just showing static content with a good cache-control header will solve the issue. If you have a restrictive IP provider, use a CDN to do it for you for cheap.
The more involved recommendation is trimming out features of hosted apps that aren't all that useful and are causing problems. A simple example of what I mean...
---
The author here is noting that his Gitea instance is seeing huge load from 20r/s, which just isn't reasonable (I actually host a Gitea instance myself and I know it can handle 10 times this traffic, even when running on a raspberry pi). So why is his failing?
Well - it sounds like he's letting bots hit every url of a public instance. Not the choice I'd make, but also not unreasonable, hosting public things is fine.
Buuut - it also sounds like he's left the "Download archive" button enabled on the instance.
That's not a good call. It's a feature that's used very rarely by real humans, but is tripmine for any bot scanning the site to trigger high load and network traffic.
Want a 5 second solution to the problem? Set `DISABLE_DOWNLOAD_SOURCE_ARCHIVES` in the Gitea config (see https://docs.gitea.com/administration/config-cheat-sheet). Problem solved. Bots are no longer an issue. They are welcome to scan and not cause problems anymore.
What if your app doesn't have an easy config option? Nginx will happily help, with far less complexity and frustrations than trying to blindly play whack-a-mole with IP addresses (this is terrible and does not work... period).
Configure nginx with a specific path rule that either blocks requests to that path entirely, or places it behind basic auth (it doesn't need to be clever, and you don't even need to make it secret - hell, put the basic auth user/pass directly in the repo's readme, or show it on your site.) The bots won't hit it anymore.
---
So I guess what I'm saying is really - if you're finding that bots on your sites are causing a problem, consider just treating them like users and solving the problem, instead of going mad and trying to remove the bots.
Be constructive instead of destructive.
Ultimately, a lot of those bots are scanning that content to show to users, many are even doing it directly at the request of the user currently interacting with them.
Falling into the trap of the "fuck the bots" mindset is a sure way to lose (although it can feel good emotionally). It's not understanding the problem, it's not solving the problem, and it's limiting access to a thing you intentionally made public. Users are on the other end of those bots.
He's choosing to play the "everyone loses" square of the prisoner's dilemma.
> My biggest recommendation is to just get familiar with the caching constructs that are available.
That doesn't help. They request seemingly random resources in seemingly random order. While they do often hit some links multiple times it's usually too few and far between for caching to be of any help.
As to the rest, "Just turn features off. No one uses them, trust me bro!"
