Curl Command Examples and Usage

cURL is awesome tool and it comes by default installed most of the distributions. If you don’t have then you can simply installed it using yum command or apt-get command.

cURL supports various protocols such as, DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP.

In this article I will provide you 10 practical cURL usage examples:

1. Download a Single File

Use the below command to get the content of the URL and display it in the STDOUT on your terminal.

# curl http://www.looklinux.com

If you want to store the output in a file you can redirect it as shown below. It will also display some additional information regarding download statistics.

# curl http://www.looklinux.com > looklinux.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 39514    0 39514    0     0  60002      0 --:--:-- --:--:-- --:--:-- 72636

2. Save the cURL Output In a File

You can also save the curl output to a file using -o/-O options.

–o (Lowercase) the output will be saved in the filename provided in the command line.
–O (Uppercase) the filename in the URL will be taken and it will be used as the filename to store the output.

# curl -o looklinux.html  http://www.looklinux.com/robots.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
102  3689    0  3689    0     0   443k      0 --:--:-- --:--:-- --:--:--  514k

Now the page robots.txt saved in the file named ” looklinux.html “. You can also see the progress meter for the download.

If you use curl -O (Uppercase) it will save the content in the file named ‘gettext.html’ itself in the local machine.

# curl -O http://www.looklinux.com/robots.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
102  3689    0  3689    0     0   594k      0 --:--:-- --:--:-- --:--:--  720k

You can also use wget to download files. Refer to wget examples.

3. Download Multiple Files at a Time

You can download multiple files in a single shot by specifying the URLs on the command line.

# curl -O http://www.gnu.org/software/gettext/manual/html_node/index.html -O http://www.gnu.org/software/gettext/manual/gettext.html

The above command will download both index.html and gettext.html and save it in the same name under the current directory.

Note: When you download multiple files from a same server, curl will try to re-use the connection.

4. Follow HTTP Location Headers

Curl does’t not follow the HTTP location headers by default. It is also termed as redirects. When a requested web page is moved to another place, then an HTTP Location header will be sent as a Response and it will have where the actual web page is located.

For example, when someone types google.com in the browser from India, it will be automatically redirected ‘google.co.in’. This is done based on the HTTP Location header.

# curl http://www.google.com
<TITLE>302 Moved</TITLE>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.co.in/">here</A>

Above you can see that the requested document is moved to “http://www.google.co.in/”.

5. Resume Previous Download

You can continue or resume download using curl -C option which was stopped already for some reason. This will be helpful if you download large files and the download got interrupted.

For example, now start downloading big file using curl and press Ctrl+C to stop it in between the download.

# curl -O http://www.gnu.org/data/download/manual/myfile.html
##############             20.1%

# displayed a progress bar instead of a progress meter.

Above download was stopped at 20.1%. We can continue the download the using curl -C. Download continues from 20.1%.

# curl -C - -O http://www.gnu.org/data/download/manual/myfile.html
###############            21.1%

6. Set Limit The Data Transfer Rate

You can set the limit the amount at which the data gets transferred using –limit-rate option.

# curl --limit-rate 1000B -O http://www.gnu.org/data/download/manual/myfile.html

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  1 1215k    1 13601    0     0    957      0  0:21:40  0:00:14  0:21:26   999
  1 1215k    1 14601    0     0    960      0  0:21:36  0:00:15  0:21:21   999
  1 1215k    1 15601    0     0    962      0  0:21:34  0:00:16  0:21:18   999

Above is the progress meter. You can see that the current speed is near to the 1000 bytes.

7. Download File If Modified Before/After The Given Time

Using -z option you can download the file that are modified after a particular time in curl. It works for both HTTP and FTP.

# curl -z 20-Jul-17 http://looklinux.com/xy.html

The above command will download the yy.html only if it is modified later than the given date and time

# curl -z -20-Jul-17 http://looklinux.com/xy.html

The above command will download the yy.html, if it is modified before than the given date and time.

8. Pass HTTP Authentication in Curl

If any websites require a username and password to view the content you can use curl -u option to get its content. You can pass credentials from CURL to the web server like below:

# curl -u username:password URL

9. Download File From FTP Server

You can also download file from ftp servers. If FTP path is a directory, by default it will list the files under the specific directory.

# curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/xyz.html

Above command will download the xyz.html file from the ftp server and save it in the locat directory.

# curl -u ftpuser:ftppass -O ftp://ftp_server/public_html/

Above url will list all the files and directories under the given URL

10. Use Proxy To Download A File

You can specify CURL to use proxy to do the specific operation using -x option. You will need hostname and port of the proxy.

# curl -x proxysever.test.com:3128 http://google.co.in

Thanks:)