Curl

Curl#

This page is a specific guide to the curl Linux utility. The following cell starts the docker container with httpbin - http requests mirror, that we’ll use to play with curl.

docker run --rm -itd --name example_server -p 80:80 kennethreitz/httpbin
17bb3c56ab5ec3a3877a6dd84b871cf2f6b690e736f814ab312a19856d2acdf1

Note don’t forget to stop the container after all:

docker stop example_server

curl can work with different protocols. The behaviour of the program is different for different protocols.

curl --help categories
Usage: curl [options...] <url>
Invalid category provided, here is a list of all categories:

 auth        Different types of authentication methods
 connection  Low level networking operations
 curl        The command line tool itself
 dns         General DNS options
 file        FILE protocol options
 ftp         FTP protocol options
 http        HTTP and HTTPS protocol options
 imap        IMAP protocol options
 misc        Options that don't fit into any other category
 output      Filesystem output
 pop3        POP3 protocol options
 post        HTTP Post specific options
 proxy       All options related to proxies
 scp         SCP protocol options
 sftp        SFTP protocol options
 smtp        SMTP protocol options
 ssh         SSH protocol options
 telnet      TELNET protocol options
 tftp        TFTP protocol options
 tls         All TLS/SSL related options
 upload      All options for uploads
 verbose     Options related to any kind of command line output of curl

To get help for a specific protocol, use the following syntax: curl --help <protocol-name>. The following cell shows help for dns protocol.

curl --help dns
Usage: curl [options...] <url>
dns: General DNS options
     --dns-interface <interface> Interface to use for DNS requests
     --dns-ipv4-addr <address> IPv4 address to use for DNS requests
     --dns-ipv6-addr <address> IPv6 address to use for DNS requests
     --dns-servers <addresses> DNS server addrs to use
     --doh-cert-status    Verify the status of the DoH server cert via OCSP-staple
     --doh-insecure       Allow insecure DoH server connections
     --doh-url <URL>      Resolve host names over DoH
 -4, --ipv4               Resolve names to IPv4 addresses
 -6, --ipv6               Resolve names to IPv6 addresses

Pass headers#

To pass headers in the curl command, we need to use the -H option. The full syntax should be curl -H "<header1 name>: <header1 value>" ....


In the following example, we’ll pass additional headers to the curl command.

curl \
    -H "new-header1: new-value" \
    -H "new-header2: new-value" \
    localhost:80/anything
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "localhost", 
    "New-Header1": "new-value", 
    "New-Header2": "new-value", 
    "User-Agent": "curl/7.81.0"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "172.17.0.1", 
  "url": "http://localhost/anything"
}

As a result, there are headers in the output.

Get headers#

There are two ways to print headers using curl:

  • The -I/--head option tells curl to download only the headers and print them to the standard output stream.

  • The -D option specifies a file where the headers will be saved, but both the headers and the body of the response will be downloaded.


The following cell demonstrates how headers can be retrieved using the -I option.

curl -I localhost:80
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Tue, 10 Sep 2024 08:32:30 GMT
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 9593
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

The following cell demonstrates how headers can be retrieved using the -D option, with headers redirected to the standard output (using - after the option to specify that).

curl -D - localhost:80/anything
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Tue, 10 Sep 2024 08:32:14 GMT
Connection: keep-alive
Content-Type: application/json
Content-Length: 267
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "localhost", 
    "User-Agent": "curl/7.81.0"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "172.17.0.1", 
  "url": "http://localhost/anything"
}