Curl#
This page is a specific guide to the curl
Linux utility. The following cell starts the docker container with httpbin - http requests mirror, that we’ll use to play with curl.
docker run --rm -itd --name example_server -p 80:80 kennethreitz/httpbin
17bb3c56ab5ec3a3877a6dd84b871cf2f6b690e736f814ab312a19856d2acdf1
Note don’t forget to stop the container after all:
docker stop example_server
curl
can work with different protocols. The behaviour of the program is different for different protocols.
curl --help categories
Usage: curl [options...] <url>
Invalid category provided, here is a list of all categories:
auth Different types of authentication methods
connection Low level networking operations
curl The command line tool itself
dns General DNS options
file FILE protocol options
ftp FTP protocol options
http HTTP and HTTPS protocol options
imap IMAP protocol options
misc Options that don't fit into any other category
output Filesystem output
pop3 POP3 protocol options
post HTTP Post specific options
proxy All options related to proxies
scp SCP protocol options
sftp SFTP protocol options
smtp SMTP protocol options
ssh SSH protocol options
telnet TELNET protocol options
tftp TFTP protocol options
tls All TLS/SSL related options
upload All options for uploads
verbose Options related to any kind of command line output of curl
To get help for a specific protocol, use the following syntax: curl --help <protocol-name>
. The following cell shows help for dns
protocol.
curl --help dns
Usage: curl [options...] <url>
dns: General DNS options
--dns-interface <interface> Interface to use for DNS requests
--dns-ipv4-addr <address> IPv4 address to use for DNS requests
--dns-ipv6-addr <address> IPv6 address to use for DNS requests
--dns-servers <addresses> DNS server addrs to use
--doh-cert-status Verify the status of the DoH server cert via OCSP-staple
--doh-insecure Allow insecure DoH server connections
--doh-url <URL> Resolve host names over DoH
-4, --ipv4 Resolve names to IPv4 addresses
-6, --ipv6 Resolve names to IPv6 addresses
Pass headers#
To pass headers in the curl command, we need to use the -H
option. The full syntax should be curl -H "<header1 name>: <header1 value>" ...
.
In the following example, we’ll pass additional headers to the curl
command.
curl \
-H "new-header1: new-value" \
-H "new-header2: new-value" \
localhost:80/anything
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Host": "localhost",
"New-Header1": "new-value",
"New-Header2": "new-value",
"User-Agent": "curl/7.81.0"
},
"json": null,
"method": "GET",
"origin": "172.17.0.1",
"url": "http://localhost/anything"
}
As a result, there are headers in the output.
Get headers#
There are two ways to print headers using curl
:
The
-I/--head
option tellscurl
to download only the headers and print them to the standard output stream.The
-D
option specifies a file where the headers will be saved, but both the headers and the body of the response will be downloaded.
The following cell demonstrates how headers can be retrieved using the -I
option.
curl -I localhost:80
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Tue, 10 Sep 2024 08:32:30 GMT
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 9593
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
The following cell demonstrates how headers can be retrieved using the -D
option, with headers redirected to the standard output (using -
after the option to specify that).
curl -D - localhost:80/anything
HTTP/1.1 200 OK
Server: gunicorn/19.9.0
Date: Tue, 10 Sep 2024 08:32:14 GMT
Connection: keep-alive
Content-Type: application/json
Content-Length: 267
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Host": "localhost",
"User-Agent": "curl/7.81.0"
},
"json": null,
"method": "GET",
"origin": "172.17.0.1",
"url": "http://localhost/anything"
}