I’ve found that all the web archiving software I’ve encountered are either manual (you have to archive everything individually in a separate application) or crawler-based (which can end up putting a lot of extra load on smaller web server, and could even get your ip blocked).
Are there any solutions that simply automatically archive web pages as you load them in your browser? If not, why aren’t there?
I could also see something like that being useful as a self-hosted web indexer, where if you ever go “I think I’ve seen this name before”, you can click on it, and your computer will say something like “this name appeared in a news headline you scrolled past two weeks ago”
You must log in or # to comment.
The Firefox extension for archive.org has an option to archive the page you visit if said page hasn’t been archived recently. Its not exactly what you’re asking for, but similar
Check out
archive warriorIt’s dead simple to set up on Docker, and will run in the background while you help literally save the internet. Ignore the steps about watchtower, as that has been deprecatedWait, watchtower is deprecated? Noooooo.
https://github.com/containrrr/watchtower/discussions/2135
Dang. Its still working fine for me for now (just like that long deprecated trailer downlaoder for the arrs).
Does archive warrior have a way of downloading pages as you visit them in a browser? I read the link but I only saw references to following Archive Team’s tasks.
I think a squid proxy can do something like that, or could be tweaked to do that, if you really wanted to.
web pages used to sort of operate that way with the ‘Temporary Internet Files’ folder. i’m not sure how it’s changed i just know this was how i used to circumvent websites that disabled right-clicking to save their images.
Maybe offpunk could fit? I’ve never used it but I read the blog post about it
There’s a Firefox extension which makes a full-text index of every page you visit - it seems to work, but I found the search a bit unreliable so I stopped using it: https://addons.mozilla.org/en-US/firefox/addon/full-text-tabs-forever/
wget is the command line program to do what you’re saying. Or what I use, anyway. Not tied to a browser, though. Maybe you could export your history and pipe it into wget if you’re using Linux or have a Linux-like command line?
I also use the FF SingleFile plugin. Again, not automatic, though.
deleted by creator
