Install and use Wget command like a pro (13 tips)
What is Wget?
Wget is a free software package for retrieving files using HTTP, HTTPS, FTP and FTPS, the most widely used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc.
How to install Wget?
# Install wget in Ubuntu \ Debian Linux
apt-get install wget
# Install wget on RHEL / CentOS / Fedora
yum install wget
# Install wget on OpenSUSE
zypper install wget
# Install wget on ArchLinux
pacman -Sy wget
# Install wget on FreeBSD
pkg install wget
# Install wget Using FreeBSD Ports Collection
portsnap fetch update
cd /usr/ports/ftp/wget
make install clean
rehash ## or hash -r for 'bash/sh/ksh'
How to use Wget?
- Now if we want do download a single file using HTTP we type:
# Download a single file
wget http://site-name.com/file-name.tar.gz
- You can download a web page and save it to a folder of your desire:
Download a website
wget -o index.html http://site-name.com/page-url
Download a website into a different folder
wget --directory-prefix='./home/user/Downloads/site-name/' http://site-name.com/page-url
3. Here is how to download files from password protected websites:
Download files from password protected websites
wget ‐‐http-user=username ‐‐http-password=password http://site-name.com/path-secret/file.tar.gz
4. Another way is to download specific type of files from the website:
Download specific type of files from the website
This will download all the mp3 files
$ wget --level=2 --recursive --accept mp3 http://site-name.com
will download all jpeg files
$ wget ‐‐level=1 ‐‐recursive ‐‐no-parent ‐‐accept jpg,JPG http://site-name.com/
5. A cool options is to download multiple files with different protocols:
Download multiple files with different protocols
wget http://site-name.com/file.tar.gz ftp://151.232.45.6/picture.jpg
6. It's also possible to limit the bandwidth of a file you are downloading:
Limit the bandwidth of a file you are downloading
wget --limit-rate=50k http://site-name.com/file.rar
7. You can download a entire website if you wish with all the files and folders inside:
Mirror entire websites (all its pages and assets)
wget --mirror --no-parent --continue http://site-name.com
8. You can put URLs in a file and then tell Wget to download all of the links in the file Download all of the links in the file
wget ‐‐input filename.txt
9. Here is how to resume a currently download file from where it was left
Resume a currently download file from where it was left
wget -c http://site-name.com/file.zip
10. Download in the Background Using wget -b
Download in the Background Using wget -b
wget -b http://www.site-name.com/link/filename.tar.bz2
11. Check the status of the download using tail -f
Check the status of the download using tail -f
tail -f wget-log
Saving to: `filename.tar.bz2.4' 0K .......... .......... .......... .......... .......... 1% 65.5K 57s 50K .......... .......... .......... .......... .......... 2% 85.9K 49s 100K .......... .......... .......... .......... .......... 3% 83.3K 47s 150K .......... .......... .......... .......... .......... 5% 86.6K 45s 200K .......... .......... .......... .......... .......... 6% 33.9K 56s 250K .......... .......... .......... .......... .......... 7% 182M 46s 300K .......... .......... .......... .......... .......... 9% 57.9K 47s
12. A nice trick is to test Download URL using Wget –spider
Test download
wget --spider http://site-name.com/link/file.tar.bz2
13. And finally here is how to download only certain file types using wget -r -A
Download only certain file types
wget -r -A.pdf http://site-name.com/files-folder/
In this lesson, we learned some of the coolest ways to handle Wget. A tool without which no system administrator can work. For additional examples you can refer to the man page of the program.
Here is a quick video install on ubuntu and download a entire web page
https://youtu.be/-XdW8o4AD_A
We hope you enjoyed this <a href="https://lateweb.info/category/linux/">article</a>. if that is so please rate this page with the stars bellow and subscribe to our <a href="https://www.youtube.com/channel/UCh7Q9uaAt5-Z2lCZXX3OsvQ">YouTube channel</a>.