Hacker Times
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
sltkr
3 months ago
|
parent
|
context
|
favorite
| on:
The bot situation on the internet is worse than yo...
Looks like Anubis is also blocking robots.txt which seems to defeat the point of having robots.txt in the first place.
sltkr
3 months ago
[–]
Update: looks like this has been fixed now.
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
