Linux Hint Posted October 3, 2020 Share Posted October 3, 2020 Curl is a popular command-line tool used for downloading files from the Internet. It is a lightweight tool that is available on any UNIX system. Curl supports a wide range of protocols, for example, HTTP, HTTPS, FTP, FTPS, SFTP, etc. If no protocol is specified, curl defaults to HTTP. The functionalities of curl come from libcurl. Curl started its journey back in the mid-1990s when the Internet was still a new thing. Daniel Stenberg, a Swedish programmer, started the project that eventually became curl. He aimed to develop a bot that would download currency exchange rates from a webpage periodically and provide Swedish Kronor equivalents in USD to IRC users. The project was successful and, thus, curl was born. Over time, curl was further improved with the addition of new internet protocols and features. In this guide, check out how to use curl to download a file. Installing curl Today, you will find curl pre-installed in most of the Linux distros. Curl is quite a popular package and is available for any Linux distro. However, there is no guarantee that curl is currently installed in your distro. Run the command according to your distro type to install curl on your system. To install curl on Debian/Ubuntu and derivatives, enter the following: $ sudo apt install curl To install curl on RHEL, CentOS, Fedora, and derivatives, enter the following: $ sudo yum install curl To install curl on OpenSUSE and derivatives, enter the following: $ sudo zipper install curl To install curl on Arch Linux and derivatives, enter the following: $ sudo pacman -Sy curl Curl is open-source software. You can grab the curl source code and compile it manually. However, this process is more complex and should be avoided if you intend to use curl for more than testing or redistributing/packaging. The following process was demonstrated in Ubuntu. For an in-depth guide on compiling curl, check out the official curl documentation. Download the curl source code here. I have grabbed the latest version of the curl source code. At the time of writing this article, the latest version is curl v7.72.0. $ wget https://curl.haxx.se/download/curl-7.72.0.tar.xz Extract the archive. $ tar -xvf curl-7.72.0.tar.xz Run the configuration script. $ ./configure Start the compilation process. $ make -j$(nproc) Finally, install the curl program that we just compiled. $ sudo make install Using curl To demonstrate the usage of the curl program, first, we need a dummy file to download. Any online file will work for this, as long as you have the direct download link. For this guide, I will use the small file provided by think broadband. Curl Version Check out the version of curl by entering the following: $ curl --version Download File Using curl This is a very basic way of using curl. We will download the dummy file. Here, the “-O” flag tells curl to download and save the file in the current directory. $ curl -O http://ipv4.download.thinkbroadband.com/10MB.zip To download and save the file with a different file name, use the “-o” flag. With this flag, the file will be downloaded and saved at the current working directory. $ curl -o demo.file http://ipv4.download.thinkbroadband.com/10MB.zip Download Multiple Files Need to download multiple files? Follow the command structure shown below. Use either “-o” or “-O” as necessary. $ curl -O <url_ file_1> -O <url_file_2> Progress Bar By default, curl does not show any progress bar. To enable the progress bar, use the “-#” flag. $ curl -# -O http://ipv4.download.thinkbroadband.com/10MB.zip Silent Mode If you want curl to print no output, use the “–silent” flag. $ curl --silent -O http://ipv4.download.thinkbroadband.com/10MB.zip Speed Limit Curl allows you to limit the download speed. Use the “–limit-rate” flag, followed by the bandwidth limit, to do so. Here, the download speed is limited to 1mb. $ curl --limit-rate 1m -O http://ipv4.download.thinkbroadband.com/10MB.zip Manage FTP Server It is also possible to manage an FTP server using curl. Assuming that the FTP server is protected, you will need to use the “-u” flag, followed by the username and password. If no file is specified, curl will print a list of all the files and directories under the user’s home directory. $ curl -u <username>:<password> ftp://exmaple.com/ Downloading files from an FTP server is like the method shown before. However, assuming the FTP server requires user authentication, use the following command structure: $ curl -u <username>:<password> ftp://exmaple.com/<file> To upload a file to the FTP server, use the following command structure: $ curl -T <file_to_upload> -u <username>:<password> ftp://exmaple.com/ User Agent In certain situations, the URL that you are trying to access may be blocked due to a lack of a proper user agent. Curl allows you to define the user agent manually. To do so, use the flag “-A,” followed by the user agent. As for the user agent, you can use the User Agents randomizer. If you want a custom user agent, then you can find one from WhatIsMyBrowser. $ curl -A "<user_agent>" -O http://ipv4.download.thinkbroadband.com/10MB.zip Final Thoughts Despite it being a simple and lightweight tool, curl offers tons of features. Compared to other command-line download managers, like wget, curl offers a more sophisticated way of handling file downloads. For in-depth information, I always recommend checking out the man page of curl, which you can open with the following command: $ man curl Check out some of the best download managers for Linux here. Happy computing! View the full article Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.