Wget

GNU Wget
GNU Wget
Type of	download manager
Author	Hrvoje Nikšić
Developers	Mauro Tortonesi, Giuseppe Scrivano et al.
Written on
Interface	command line
operating system	Linux and others. UNIX-like , Windows
Interface languages	Russian and 38 more languages
First edition
Latest version	;
File Formats Created
condition	active
License
Site	gnu.org/software/wget/

Wget ^[4] - (GNU Wget) is a free, non - interactive console program for downloading files over a network. It supports HTTP , FTP and HTTPS , and also supports work through an HTTP proxy server . The program is included in almost all GNU / Linux distributions .

Wget is a non-interactive program. This means that after its launch, the user can affect its operation only with the help of process control tools of the operating system. As a rule, the keyboard shortcuts Ctrl + C are used to interrupt the program and Ctrl + Z to place the current job in the background. Modern web browsers usually have the function of downloading files, however, since the browser is designed for interactive operation, downloading a large number of files manually can be tedious. Browsers, as a rule, do not provide tools for automating such tasks. Wget, for example, supports loading the URLs specified in the file. This way you can make a list of files, and download them using wget at any convenient time. The command line interface allows you to control wget from other programs and scripts , which is used to automate file downloads (regular updates, monitoring server availability, etc.).

Wget allows you to upload any files on the World Wide Web (including ( X ) HTML pages) via http and https protocols, as well as files and directory lists using ftp protocol.

Files can be downloaded recursively via links in HTML pages, both from one site with a certain depth of following links, and from several. In addition, when downloading via ftp, files can be downloaded “by mask” of the name (that is, you can specify a group of files using “*”).

Wget supports downloading a file in case of a connection failure.

The development of the continuation Wget - Wget2 ^[5] is underway.

Content

Examples

Download all URLs specified in a local or external FILE:

  wget -i FILE

Downloading files to the specified directory (-P) ^[6] :

  wget -P / path / for / save ftp://ftp.example.org/some_file.iso

Usage of user name and password on FTP / HTTP (option 1):

  wget ftp: // login: password@ftp.example.org/some_file.iso

Usage of user name and password on FTP / HTTP (option 2):

  wget --user = login --password = password ftp://ftp.example.org/some_file.iso

Download in the background (-b):

  wget -b ftp://ftp.example.org/some_file.iso

Continue (-c continue) downloading a previously incomplete file:

  wget -c http://example.org/file.iso

Download the contents of the http://example.org/~user/my-archive/ directory (inaccessible link) and all its subdirectories, without moving up the directory hierarchy above:

  wget -r --no-parent http://example.org/~user/my-archive/

Server authentication is also supported:

  wget --save-cookies cookies.txt \
 --post-data 'user = foo & password = bar' \
 http://example.org/auth.php

Download the whole site (recursion depth - 10):

  wget -r -l 10 -k -o log-file.txt -p http://example.org/

 -r, --recursive enable recursive loading
 -l, --level = NUMBER recursion depth ( inf and 0 - infinity )
 -k, --convert-links make links local in loaded HTML or CSS
 -o, --output-file = FILE write messages ( logs ) to FILE
 -p, --page-requisites download all the images, etc. needed to display an HTML page
 -m, --mirror a short parameter equivalent to -N -r -l inf --no-remove-listing.

Print the contents of the response to the console:

  wget http://example.org --quiet -O -

Return Status

Prior to version 1.12, it returns 0 on success and 1 on failure. Starting from version 1.12 ^[7] , in case of an error, returns from 1 to 8, depending on the type.

Criticism

Wget development is slow, many new extensions to the HTTP, FTP , JavaScript scripts and other functions are not supported.

In some cases, cURL may be a good alternative, and rsync is usually used to create site mirrors (which cURL does not know ^[8] ). In version 1.17 and above, support for SSE2 instructions by the processor is mandatory.

wget2

The next major update to Wget should happen in wget2, which is supposed to replace it. wget2 has a significant number of improvements over Wget, primarily related to performance:

HTTP / 2 support;
HTTP compression
parallel connections;
TCP Fast Open;

and a number of others.

The main developer of wget2 is Tim Rühsen, who is also one of the main developers and current Wget maintainers.

Notes

↑ Wget homepage.
↑ Shah D. wget-1.20.3 released [stable ] - 2019.
↑ The wget Open Source Project on Open Hub: Languages Page - 2006.
<a href=" https://wikidata.org/wiki/Track:Q124688 "> </a> <a href=" https://wikidata.org/wiki/Track:P1972 "> </a>
↑ By virtue of the conventions adopted in * nix- systems on the command line, this program is called by a word beginning with a lowercase letter: wget
↑ wget2 on gitlab
↑ GNU wget manual
↑ wget manual Return Status (en)
↑ cURL FAQ : “Curl is not a web site mirroring program.”

Links

[1] Wget homepage.

[_8277764b3ca57ef7-2] Shah D. wget-1.20.3 released [stable ] - 2019.

[_c0287b3133bbeb2c-3] The wget Open Source Project on Open Hub: Languages Page - 2006.
<a href=" https://wikidata.org/wiki/Track:Q124688 "> </a> <a href=" https://wikidata.org/wiki/Track:P1972 "> </a>

[4] By virtue of the conventions adopted in * nix- systems on the command line, this program is called by a word beginning with a lowercase letter: wget

[5] wget2 on gitlab

[6] GNU wget manual

[7] wget manual Return Status (en)

[8] URL FAQ : “Curl is not a web site mirroring program.”