for a short time i had warcprox sitting behind my firefox and auto feeding its o... | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

		hexagonwin 17 days ago \| parent \| context \| favorite \| on: Google broke reCAPTCHA for de-googled Android user... for a short time i had warcprox sitting behind my firefox and auto feeding its output to pywb, it seemed to work but i had connections failing randomly after having warcprox running for more than a few hours~days. not sure if it's an issue with pywb or warcprox but there were some urls missing that i did browse on firefox, and many dynamic pages couldn't be replayed at all.

sunshine-o 17 days ago [–]

I am not surprised...

I am unfamiliar with web caching proxies like squid [0] but I am wondering if that might be the most straightforward way to do this.

So use squid and then have a batch job that go through /var/spool/squid every day and update your web archive according to some defined filters.

- [0] https://www.squid-cache.org/

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact