bash - Parallel curl (or wget) downloads

Question

(Closed.) This question does not meet Stack Overflow guidelines. It is not currently accepting answers. question from:https://stackoverflow.com/questions/66056295/parallel-curl-or-wget-downloads

Answer 1

If you're okay with a (slightly) different tool, may I recommend using GNU Wget2? It is the spiritual successor to GNU Wget. It is already available in the Debian and OpenSUSE repositories and on the AUR

Wget2 provides multi-threaded downloads out of the box with a nice progress bar to view the current status. It also supports HTTP/2 and many other newer features that were nearly impossible to add into Wget.

See my answer here: https://stackoverflow.com/a/49386440/952658 for some more details.

With Wget2, you can simply run $wget2 -i urls.txt and it will start downloading your files in parallel.

EDIT: As mentioned in the other answer, a disclaimer: I maintain both Wget and Wget2. So I'm clearly biased towards this tool