cURL security anti-patterns
Oct 15, 2020 · 969 words · 5 minute read
curl
is a widely used command line tool for interacting with HTTP resources.
People use it to download binaries from command line or to interact with HTTP API endpoints from automation scripts.
This blog shows some of the common curl
security anti-patterns and how to avoid them.
I assume you are using curl
to interact with resources via HTTP(S) protocol.
I also assume you are familiar with basic usage of the curl
.
If you have any comments or suggestions about this post, please leave a comment at the following HackerNews thread: https://news.ycombinator.com/item?id=24787438
Quick links to security anti-patterns discussed here:
- Ignoring TLS certificate validation
- Passing secrets on a command line
- Following redirects
- Storing cookies
- Downloading and executing scripts
Ignoring TLS certificate validation
Mitigation short summary
- Don’t use
-k
option in production. - Use
--cacert
to validate certificates issued by private CA. - Install private issuer CA on a system, ONLY if you COMPLETELY trust the CA issuer.
Mitigation details
Anyone who used curl
with HTTPS knows that -k
is a magic curl
flag to make any HTTPS connection “work”.
While you might view this as a harmless habit or even shortcut to get things done fast
, please don’t do it.
Even though using -k
is the fastest remedy to resolve issues with TLS, it shouldn’t be your first choice.
Especially on the production systems. You should investigate why curl
isn’t validating server’s certificate.
Here are just some reasons why certificate validation might fail and how to address them:
- Certificate has expired. Solution: Contact the server admin to fix the certificate or provide you alternative.
- Certificate is issued by private CA. Solution: If you trust issuer, download the CA certificate chain and point
curl
to it with--cacert
option. - Other certificate error. You should always investigate and understand the cause of certificate validation error, before using
-k
option.
Passing secrets on a command line
Mitigation short summary
- Use
-K
option to pass configuration file with secrets - (or) Set secrets as environment variables (and temporarily disable shell history)
Mitigation details
As with any CLI tool, you have to be careful what you leave behind in the history file of your shell. This is not a big problem on a machine where you are the only user, but on shared systems this becomes security liability.
One way to mitigate this problem is to use a config file. Config file should have appropriate file permissions set, in order to restrict access only to the users/groups you trust.
~$ curl -K /path/to/curl/config/file https://example.com/login
Another good alternative is to set secrets as environment variables.
While you are setting environment variables, make sure you temporarily disable shell history.
This is a good practice for any command, that takes secrets from the command line.
After you are done setting the environment, you can enable history back again and use the variables with the curl
command’s arguments.
~$ set +o history
~$ export API_KEY=s3cr3tKey
~$ set -o history
~$ curl -H "X-API-KEY: ${API_KEY}" https://example.com/api/resource
Some shells will not place command into shell history, if you prefix the command with single space character. This can be useful if you are setting only one or two environment variables.
Following redirects
Mitigation short summary
- Don’t use
-L
without making sure you trust full redirect chain. - Don’t use
--location-trusted
without making sure you trust full redirect chain.
Mitigation details
Sometimes resource you are trying to access can be moved under new URI, or to another server.
In this case the server will send you a redirect status code (3XX
) and a new location in the headers.
curl
has flag -L
that will automatically follow these redirections.
When using -L
you must be careful where you are getting redirected.
You shouldn’t automatically add -L
flag to all curl
calls.
If you do, you could potentially download malicious code/file, that was
If resource was moved permanently, you should update URI you are calling instead of using -L
option.
When sending username and password details via curl
to the remote server, make sure you understand which hosts it traverses before using --location-trusted
.
This option will allow sending the username and the password to all hosts that server might redirect to.
Storing cookies
Mitigation short summary
- Don’t store cookies in files, that don’t have proper access rights.
Mitigation details
Some HTTP resources require cookies to operate. To store them you can use --cookie-jar <filename>
option.
Keep in mind that cookies are as sensitive information as passwords and API keys.
It’s important that the file in which you are storing cookies has proper permissions set and isn’t readable by non-trusted users/groups.
Downloading and executing scripts
Mitigation short summary
- Don’t automatically pass a downloaded resource file to be executed by a shell.
Mitigation details
Following anti-pattern was brought to my attention by a coworker, thanks.
To make a download, installation and setup of a tool more convenient, some vendors opt to distribute installation scripts via HTTP API endpoint.
It is expected that user will download the script using curl
and pipe it to a shell for execution:
~$ curl https://eve.example.org/get-tool-script | bash
If it’s not obvious, this can cause security problems if vendor’s infrastructure is compromised. Optionally, if vendor changes the script and unwillingly introduces security misconfiguration, it will be propagated to your infrastructure.
When using such scripts for installation and configuration, do the following:
- Download the script manually and inspect it’s contents. Make sure you understand everything that script does.
- Calculate
sha256sum
of the script (or some other hash, but not SHA1 or MD5). - Use the calculated hash in the subsequent runs to ensure contents of the script didn’t change.
You automation script then might look something like this:
~$ curl -O https://eve.example.org/get-tool-script
~$ if [[ $(sha256sum get-tool-script | grep <sha256sum-that-you-calculated-earlier>) ]]; then \
bash get-tool-script; \
else \
echo "Invalid SHA256 sum. Please check the script contents!"; \
fi