How to Download all Files From a Website [10 Free Ways]
There are times when you will end up on a web page that doesn’t look like a webpage but instead looks like a folder and you can only find files being listed. This is because the webserver directory index file (index.html, default.asp and etc) is not present in the folder and the directory listings option in the webserver is turned on.
One of the reasons to offer directory listings is to provide a convenient way for the visitor to quickly browse the files in the folders and allow them to easily download the files to their computer. Sometimes directory listings are accidental due to webmasters who forget to include a .htaccess or a blank index file to prevent all the files from being listed.
To download a file, you can either left click on it or right click and select “Save link as” in your favorite browser like Firefox or Chrome. However, if you need to download multiple or even all of the files from the directory including the subfolders automatically, you will need third party tools to help you achieve that. Here are 5 different methods that you can use to download all files from a folder on a website.
Download Files With A Download Manager
If you are a frequent downloader, you probably already have a download manager program installed. Some of the popular and feature rich download managers like JDownloader are even open source software.
1. JDownloader
This very popular download manager is based on Java which means it’s cross platform and available for Windows, macOS and Linux.
While this program is able to download all files in a specific folder very easily it cannot recurse into sub folders. All you have to do is copy a URL to the clipboard while JDownloader is running and it will add a new package or set of packages to the Link Grabber with all the files. File types can be filtered out with checkboxes or deselected/removed manually. Note the JDownloader installer version contains adware.
Download JDownloader | JDownloader Portable (no adware on install)
2. FlashGet
This next download manager program is quite old but has a feature called Site Explorer which allows you to browse websites like in Windows Explorer. FlashGet has more recent versions than the 1.96 we’re looking at here, but we found this version handles HTTP and FTP sites better.
After installing and running Flashget go to the Tools menu > Site Explorer. Enter the URL and then you can browse through the site and download the files in any folder. If the site is using FTP, folders can also be multi selected and the files inside those folders will be downloaded. Only the files inside the root folder will download if the site is HTTP. Make sure to avoid the Google Toolbar offer during install.
Download From A Folder Or FTP With A Web Browser Extension
Popular browser extensions for downloading files in recent times have been DownThemAll! and Flashgot for Firefox. Sadly, they don’t work anymore since the change to the WebExtensions system in Firefox 57. However, there are still extensions available for both Chrome and Firefox that can download files from a website or FTP folder. Here’s a couple of selections for each.
Note: All the browser extensions below will only download the files from the root folder in the browser tab, they will not recurse into sub folders. If you select a folder from the download list it will simply download as an unknown file.
3. Chrono Download Manager (Chrome)
Chrono Download Manager is one of the most popular extensions of its type for Chrome. There are some advanced options but it’s relatively straightforward to quickly download all the files in a folder. Click the Chrono toolbar button and switch to the sniffer mode with the top right button in the window.
Then, cycle through the tabs selecting all the files with the top checkbox, check files individually or use the file type filter boxes below. Click the “Download All!” button and the files will be added to the queue and downloaded.
Download Chrono Download Manager
4. Download Master (Chrome)
This is another Chrome extension that downloads a load of files in a folder pretty easily. Download Master works in a similar way to Chrono but is a little more straightforward to use and what you see in the main window is it, there are no separate settings or options windows.
After you press the icon to open the download window all you have to do is check the file extension filter boxes, supply a custom filter or add files manually. Then press Download. As all the selection filters are in the same window it makes Download Master a bit faster to select multiple files or all files at once.
5. Simple Mass Downloader (Firefox)
Simple Mass Downloader is a pretty good option for Firefox since the classic old extensions no longer work. It also has some useful features like being able to add downloads from multiple tabs at once and add automatic folders to directly download files of certain types into specific folders.
Files do not automatically populate the resource list and you have to click the “Load Page Links” button to start. The checkbox at the top will select all files at once while the extensions or text filter boxes at the bottom will filter the list by whatever characters are entered. Files can be added to a queue or downloaded directly with the buttons at the bottom right.
Download Simple Mass Downloader
6. DownloadStar (Firefox)
The final extension we’ll talk about is DownloadStar for Firefox. It’s not much different than the options we’ve mentioned above but to make all files visible you have to click “Show media from links” at the bottom left of the window or only images will be shown.
The filter checkboxes don’t have a select all files option but simply putting an asterisk “*” into the Fast Filter box will quickly do that. The button to download the files isn’t immediately obvious but it’s located at the top right with the number of files to download in brackets.
To download files from a folder using something other than browser extensions or download managers, try the methods on Page 2.
Thank you very much for your good work
Thanks HAL9000 for your work, you made my day.
One of the best tool to listing and downloading FTP or http directory is IDM ; through IDM Grabber tool.
I would like to download a number of files that can be found under a http link which is always the same – just the number at the end changes. Is there an application/extention available that would be able to do that? thank you, John
Have tried so many applications and techniques over the years and so many of them don’t do FTP well.▪▪ the win Explorer doesn’t do well with my Android phone SD card and my Windows XP or Windows 7 so I have used the extension originally for Firefox and now also for Chrome call down them all its wonderful
VWget does work, I am using it right now to download from a folder deep within a hos with no index.html found by using ?index.of? You do have to use the right settings, it took a couple of goes, the first 2 times it tried to download most of the domain lol.
The “Chrono Download Manager” (Chrome) was removed from Store 1-2 months ago, for unknown reasons.
The “Simple mass downloader” has a better Chrome version, more frequently updated.
The good news is that the DownThemAll returned as a web extension, on both Firefox and Chrome. Not multi threading yet (if ever), but still a very good option.
The only web extension with decent multi threading seems to be “Multithreaded Download Manager”, on Firefox
As a general mass downloader, I still prefer the “Simple mass Downloader”, at least on Chrome, but DownThemAll may be the most future proof of all.
Typical really, this article was updated almost exactly 4 weeks ago and already one extension has been removed from the Chrome store and another that didn’t work has come back from the dead…
Sounds like it will have to be updated yet again in the near future.
However, if you’re dissatisfied and ready for change, Dropbox is my top choice. I’ve been a user since 2008 and never had an issue. The service is supported by a large ecosystem of apps, it’s easy to use and share files with others, and it continues to evolve in positive ways. Though some dislike its revamped Dropbox desktop app, I appreciate the extra features it puts within reach, such as the ability to quickly create G Suite files. That said, Box, Google Drive, and Microsoft OneDrive are all great choices for most users, depending upon your needs and budget.
I haven’t recommended and will not recommend Dropbox after they deleted my files and basically banned my account several years ago for doing absolutely nothing wrong.
Is there a similar software available for Android platform?
I use JDownloader
another option?
Visualwget DOES NOT WORK.
It does not download subdirectories after following your instrusctions.
Downthemall is the same..
Both are files in single directory or the .index file only.
It works, but You have to set the right settings :)
It does not…
WinHTTrack and VisualWget are crap.
Thank you a LOT. In the past I used flashget mainly, but now it is mot compatible with latest fire fox, I don’t want to risk continue using older version of FireFox.
vwget did what I needed exactly, I would not have known about the recursive feature without your help. THANK YOU!
DownThemAll is a Firefox add-on which works fine for me.
The folks at the subreddit /r/opendirectories are using Felistar (www.moonstarsky.com), a tool built by another redditor. You should check it out
Thanks for the share, please keep it updated for GNU/Linux users.
I am trying to download multiple files from a facebook group. I only need the svg files. What is my best option?
very helpful thanks bro
thanks very much for your info…
I like vwget personally…
really small and effective tool
Thanks a lot. Personally I found VisualWget to be the best option.
Super helpful!
Very Useful. Thank you.
Sweet…
Filezzila worked for me! :)
thanks mate. This was helpful!
Really appreciate this!
Good on ya!
I tried no.4 (FileZilla) and it worked like a charm. Thanks.
really helpful…..and helps save time
Superb ..it did the trick !!!
Thanks this is very useful. You are a good man.
Raymond you’re the best ! Martin C. you are great too for -parent tricks ;)
You Sir are a legend :)
thanks man
$ wget –no-parent –wait=10 –recursive –accept=zip –no-directories yoursite.com
–wait can be replaced with 1 (second) providing the server from you will download don’t kick your ass out.
Remove –no-directories to completely crawl and download everything matching your criteria (zip files here) starting from the root directory.
thank you sir its work thank you so much
good
Thanks contributor. This is very much appreciated.
thanks man, very helpful
Very helpful post
Your forum is helping me a lot..
Great!!
thanks mate you helped me a lot
Thank you Ray – very useful bits of kit.
cool! thanks ray!
thanks Raymond always good to get the latest news from you
cool. thanks. Raymond rocks!
Thanks bro, love the forum and your article
thanks ray.