Why is spending all that CPU time to scrape the handful of sites that use Anubis... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		creata 9 months ago \| parent \| context \| favorite \| on: Anubis Works Why is spending all that CPU time to scrape the handful of sites that use Anubis worth it to them?

vhcr 9 months ago [–]

Because it's not a lot of CPU, you only have to solve it once per website, and the default policy difficulty of 16 for bots is worthless because you can just change your user agent so you get a difficulty of 4.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact