How to Download all Files From a Website [10 Free Ways]
7. Wget
Wget is a free and very powerful file downloader that comes with a lot of useful features including resume support, recursive download, FTP/HTTPS support, and etc. In “The Social Network” movie, Mark Zuckerberg is seen using the Wget tool to download all the student photos from his university to create Facemash. Wget is a command line tool which can be a bit difficult to use for some basic users.
Thankfully there are free front-end GUI’s for Wget such as VisualWget that make it so much easier to use Wget by simply clicking on the checkboxes rather than manually typing the command line arguments. Download VisualWget, extract and run VisualWget.exe. Click on the New icon to open a New Download window. Enter the URL that you want to download and select the location that you want to save the files to.
If you need to download multiple folders including subfolders, go to Advanced, click on Recursive Retrieval and tick on the first checkbox “–recursive“. Finally, click OK to start downloading.
Although there are a few GUIs for Wget, we recommend VisualWget because it is free, portable, comes together with Wget and there is no need to download separately and manually unpacking it to the program’s folder or even configuring the path. Note that if you are a Windows 10 user and don’t already have .NET Framework 2/3.5 installed, you will need to download it during the VisualWget installation.
8. HTTrack
HTTrack is an offline browser that downloads the whole website for offline viewing. Software like this is capable of crawling into subfolders, downloading all or filtered files and then converting the live hyperlinks to an offline version. Some other offline browsers are Offline Explorer and Teleport but they are shareware and come with powerful parsing capabilities.
HTTrack is a good free alternative to the paid offline browsers. You can download the portable version of HTTrack, extract and run WinHTTrack.exe. Click Next, give the project a name and click Next.
Now enter or paste the URL(s) that you want to download to the Web Addresses box and click Next. If the URL requires authentication, click the Add URL button and you get to enter the login and password for the URL. Click Next and Finish. HTTrack will now start crawling the given URL and download files that it finds.
9. FileZilla
The best program to download all files and subfolders from an FTP server is of course going to be dedicated FTP client software like FileZilla. They obviously understand FTP commands and are able to crawl recursively into subfolders without problems. In FileZilla Client, all you need to do is enter the FTP address in the Host box, enter a username and password if it requires authentication, or leave it blank if not, and click the Quickconnect button.
Once you’re logged in, simply right click on a folder in the right pane and select Download which will start downloading all files and folders from the selected directory.
Please take note that FTP clients can only handle the FTP protocol and they cannot download files from HTTP web pages.
10. Windows File Explorer FTP
A lot of people will know that Windows File Explorer can access FTP sites but a lot of people also won’t know that. It’s pretty easy and quite basic but if you just want to select and download some files or folders on a site then it may be all you need and requires no extra software.
To access an FTP site all you have to do is enter its address in a File Explorer window address bar. Note that only a genuine FTP address will work and anything HTTP related won’t work. Browse and select the files and folders you want like you would in a local folder, right click and select Copy. Then Paste into the location of your choice and the files will download. A “Copy to Folder” option is also available on right click.
The useful thing about the Windows Explorer option is it will recurse into subfolders so if you select a root folder everything inside it will download. Simply close the Explorer window or browse to a local folder when you want to close the FTP connection.
Thank you very much for your good work
Thanks HAL9000 for your work, you made my day.
One of the best tool to listing and downloading FTP or http directory is IDM ; through IDM Grabber tool.
I would like to download a number of files that can be found under a http link which is always the same – just the number at the end changes. Is there an application/extention available that would be able to do that? thank you, John
Have tried so many applications and techniques over the years and so many of them don’t do FTP well.▪▪ the win Explorer doesn’t do well with my Android phone SD card and my Windows XP or Windows 7 so I have used the extension originally for Firefox and now also for Chrome call down them all its wonderful
VWget does work, I am using it right now to download from a folder deep within a hos with no index.html found by using ?index.of? You do have to use the right settings, it took a couple of goes, the first 2 times it tried to download most of the domain lol.
The “Chrono Download Manager” (Chrome) was removed from Store 1-2 months ago, for unknown reasons.
The “Simple mass downloader” has a better Chrome version, more frequently updated.
The good news is that the DownThemAll returned as a web extension, on both Firefox and Chrome. Not multi threading yet (if ever), but still a very good option.
The only web extension with decent multi threading seems to be “Multithreaded Download Manager”, on Firefox
As a general mass downloader, I still prefer the “Simple mass Downloader”, at least on Chrome, but DownThemAll may be the most future proof of all.
Typical really, this article was updated almost exactly 4 weeks ago and already one extension has been removed from the Chrome store and another that didn’t work has come back from the dead…
Sounds like it will have to be updated yet again in the near future.
However, if you’re dissatisfied and ready for change, Dropbox is my top choice. I’ve been a user since 2008 and never had an issue. The service is supported by a large ecosystem of apps, it’s easy to use and share files with others, and it continues to evolve in positive ways. Though some dislike its revamped Dropbox desktop app, I appreciate the extra features it puts within reach, such as the ability to quickly create G Suite files. That said, Box, Google Drive, and Microsoft OneDrive are all great choices for most users, depending upon your needs and budget.
I haven’t recommended and will not recommend Dropbox after they deleted my files and basically banned my account several years ago for doing absolutely nothing wrong.
Is there a similar software available for Android platform?
I use JDownloader
another option?
Visualwget DOES NOT WORK.
It does not download subdirectories after following your instrusctions.
Downthemall is the same..
Both are files in single directory or the .index file only.
It works, but You have to set the right settings :)
It does not…
WinHTTrack and VisualWget are crap.
Thank you a LOT. In the past I used flashget mainly, but now it is mot compatible with latest fire fox, I don’t want to risk continue using older version of FireFox.
vwget did what I needed exactly, I would not have known about the recursive feature without your help. THANK YOU!
DownThemAll is a Firefox add-on which works fine for me.
The folks at the subreddit /r/opendirectories are using Felistar (www.moonstarsky.com), a tool built by another redditor. You should check it out
Thanks for the share, please keep it updated for GNU/Linux users.
I am trying to download multiple files from a facebook group. I only need the svg files. What is my best option?
very helpful thanks bro
thanks very much for your info…
I like vwget personally…
really small and effective tool
Thanks a lot. Personally I found VisualWget to be the best option.
Super helpful!
Very Useful. Thank you.
Sweet…
Filezzila worked for me! :)
thanks mate. This was helpful!
Really appreciate this!
Good on ya!
I tried no.4 (FileZilla) and it worked like a charm. Thanks.
really helpful…..and helps save time
Superb ..it did the trick !!!
Thanks this is very useful. You are a good man.
Raymond you’re the best ! Martin C. you are great too for -parent tricks ;)
You Sir are a legend :)
thanks man
$ wget –no-parent –wait=10 –recursive –accept=zip –no-directories yoursite.com
–wait can be replaced with 1 (second) providing the server from you will download don’t kick your ass out.
Remove –no-directories to completely crawl and download everything matching your criteria (zip files here) starting from the root directory.
thank you sir its work thank you so much
good
Thanks contributor. This is very much appreciated.
thanks man, very helpful
Very helpful post
Your forum is helping me a lot..
Great!!
thanks mate you helped me a lot
Thank you Ray – very useful bits of kit.
cool! thanks ray!
thanks Raymond always good to get the latest news from you
cool. thanks. Raymond rocks!
Thanks bro, love the forum and your article
thanks ray.