I have, for the most of my scripts, used Wget as my primary way of downloading files. But this does have some issues, one of which I will outline below:

Once in a while I will want to download a little bit more than a single webpage. While Singlefile and the built-in page downloader in Firefox work for those, they don’t for anything more than just that single page. One thing I discovered is that if you go to the developer tools, over to the network tab, you can select a network event, and right click and select Copy Value > Copy as cURL, you can get a curl command that will download that event for you.

So, I present: CTOW, or Curl TO Wget, which is basically just a fancy find-and-replace script, that exchanges curl arguments for the same or similar wget ones. It’s semi-based on the curlconverter website, but isn’t made using javascript.

One choice I made, is that I was going to do this project in Rust, as to me, it’s a slightly more rich version of C, and it also has a richer string manipulation, while keeping C’s speed.

I started off just taking an existing curl command, parsed from the command-line and splitting it off into the arguments, and working out how to correctly handle spaces in the arguments. Then, I added a very basic find-and-replace method, that works on a single argument at a time.

At this point, I was finding the Vector<_> type in Rust quite useful, and had a working minimum viable product, at which point I decided that this could be useful to other people out there, so I added the GPLv3+ license to the source code, and had a go at uploading it to cargo, Rust’s package manager.

Enter jaysml, who was a great help, adding the errors, and handling for them, tests, both of which allowed me to learn how errors and tests work in Rust in a nicer way than just reading through the book.

After this, I added to the errors, and added extra formatting to the outputs, using ANSI escape sequences. Then, it was jaysml to the rescue again, showing me how Rust crates are typically structured, and how the rustdoc system works. I added more docs, including examples, to the new lib.rs file, and kept updating the crate throughout.

Building ctow taught me a lot about how Rust projects are structured and built, and although it’s not one-to-one in converting curl to wget arguments, it’s a good base for more to be added easily.