Heykuki News
Top
New
Best
Ask
Show
Jobs
Toggle theme
Login
Top
New
Best
Ask
Show
Jobs
Robots.txt is a suicide note
archiveteam.org
115 points
panza
15 years ago
"Archive Team interprets ROBOTS.TXT as damage and temporary madness, and works around it. Everyone should. If you don't want people to have your data, don't put it online."
91 comments
Loading...
Robots.txt is a suicide note | Heykuki News