I could really use a commandline mechanism for saving a webpage locally for offline viewing later. Saving the page in a single file is ideal.

“singlefile”, the browser extension

For GUI users there is a quite useful extension (single-file). That can be run from the CLI by installing single-file-cli. But I am not enthusiastic about the bloaty option. You must first install some obscure package manager called “Deno”, which does not exist in the official Debian repositories (thus has not proven to be worthy for mainstream public use, and also complicates every Debian upgrade). Then it uses JavaScript to instrument a bloaty GUI browser. Not great.

Worth noting that there is Org-mode integration but this does not get around the the non-Debian dependency on Deno.

wget

A page can be fetched using this command:

$ wget -U 'stopDiscriminatingAgainstWg3t' -c -P "$dir" -E -H -k -K --xattr -p "$url"
$ printf '%s' "$url" > "$dir"/url.txt

The printf is needed because wget just dumps a big tree of files with no indication of which file must be opened to render the whole page. Indeed it sucks. Saving the URL gives us a fighting chance at locating the file but it’s still a pain. And from there, the files are still scattered on your hard drive.

There is a non-Debian tool called HTMLArk. Again, I am not thrilled about non-Debian stuff. But it claims to be able to produce a single HTML file from a scattered collection of files. I have not tried it but in principle it can be used to tidy up the mess dumped by wget.

ArchiveBox and Grunt-inline

ArchiveBox looks interesting. But it’s non-Debian so not exactly spot-on. Same issue with Grunt-inline. There is some grunt stuff in the Debian repos but not this tool specifically.

webpages2html

This is a python script but the script and its dependencies are non-debian. Requires using pip which is a disaster of a tool that is barely suitable for developers but hardly suitable for the end users who end up getting pushed into it b/c there is no proper pkg manager.

What else?

Any decent option I have overlooked?

  • evenwichtOPM
    1·
    3 天前

    Glad to see it’s in the official Debian repos. That encourages me to try it and fiddle around with it.

    The landing page says: “It allows you to download a World Wide Web site from the Internet”, which gives the feeling of a 1990s project. Which isn’t bad… It suggests the dev is probably not keen to use silly modern day bloaty garbage like JS, npm, electron, python, and awkward pkg managers. And indeed it’s in C.

    Although I had to wonder how it notably differs from wget -c -E -H -k -K … Then I saw the --single-file option. So that pretty much settles it – I must check it out. This may be the perfect tool. Thanks for the tip!

    (update) I installed it. Looks promising.

    (update 2) Superficially i love the design… the way it has a server and remembers sessions. A lot of thought went into the design. But I can’t get it to work. There is a config for “proxy”; does not say whether it’s an HTTP proxy or SOCKS. It does not indicate if an URL is expected or just a hostname. My proxy setting does not stick. And no data is fetched in the end. Hard to know if respect for robots.txt is hindering it.