cURL security anti-patterns

Oct 15, 2020 · 969 words · 5 minute read

curl is a widely used command line tool for interacting with HTTP resources. People use it to download binaries from command line or to interact with HTTP API endpoints from automation scripts. This blog shows some of the common curl security anti-patterns and how to avoid them.

I assume you are using curl to interact with resources via HTTP(S) protocol. I also assume you are familiar with basic usage of the curl. If you have any comments or suggestions about this post, please leave a comment at the following HackerNews thread:

Quick links to security anti-patterns discussed here:

Ignoring TLS certificate validation

Mitigation short summary

  • Don’t use -k option in production.
  • Use --cacert to validate certificates issued by private CA.
  • Install private issuer CA on a system, ONLY if you COMPLETELY trust the CA issuer.

Mitigation details

Anyone who used curl with HTTPS knows that -k is a magic curl flag to make any HTTPS connection “work”. While you might view this as a harmless habit or even shortcut to get things done fast, please don’t do it.

Even though using -k is the fastest remedy to resolve issues with TLS, it shouldn’t be your first choice. Especially on the production systems. You should investigate why curl isn’t validating server’s certificate.

Here are just some reasons why certificate validation might fail and how to address them:

  1. Certificate has expired. Solution: Contact the server admin to fix the certificate or provide you alternative.
  2. Certificate is issued by private CA. Solution: If you trust issuer, download the CA certificate chain and point curl to it with --cacert option.
  3. Other certificate error. You should always investigate and understand the cause of certificate validation error, before using -k option.

Passing secrets on a command line

Mitigation short summary

  • Use -K option to pass configuration file with secrets
  • (or) Set secrets as environment variables (and temporarily disable shell history)

Mitigation details

As with any CLI tool, you have to be careful what you leave behind in the history file of your shell. This is not a big problem on a machine where you are the only user, but on shared systems this becomes security liability.

One way to mitigate this problem is to use a config file. Config file should have appropriate file permissions set, in order to restrict access only to the users/groups you trust.

~$ curl -K /path/to/curl/config/file

Another good alternative is to set secrets as environment variables. While you are setting environment variables, make sure you temporarily disable shell history. This is a good practice for any command, that takes secrets from the command line. After you are done setting the environment, you can enable history back again and use the variables with the curl command’s arguments.

~$ set +o history
~$ export API_KEY=s3cr3tKey
~$ set -o history
~$ curl -H "X-API-KEY: ${API_KEY}"

Some shells will not place command into shell history, if you prefix the command with single space character. This can be useful if you are setting only one or two environment variables.

Following redirects

Mitigation short summary

  • Don’t use -L without making sure you trust full redirect chain.
  • Don’t use --location-trusted without making sure you trust full redirect chain.

Mitigation details

Sometimes resource you are trying to access can be moved under new URI, or to another server. In this case the server will send you a redirect status code (3XX) and a new location in the headers. curl has flag -L that will automatically follow these redirections.

When using -L you must be careful where you are getting redirected. You shouldn’t automatically add -L flag to all curl calls. If you do, you could potentially download malicious code/file, that was If resource was moved permanently, you should update URI you are calling instead of using -L option.

When sending username and password details via curl to the remote server, make sure you understand which hosts it traverses before using --location-trusted. This option will allow sending the username and the password to all hosts that server might redirect to.

Storing cookies

Mitigation short summary

  • Don’t store cookies in files, that don’t have proper access rights.

Mitigation details

Some HTTP resources require cookies to operate. To store them you can use --cookie-jar <filename> option. Keep in mind that cookies are as sensitive information as passwords and API keys. It’s important that the file in which you are storing cookies has proper permissions set and isn’t readable by non-trusted users/groups.

Downloading and executing scripts

Mitigation short summary

  • Don’t automatically pass a downloaded resource file to be executed by a shell.

Mitigation details

Following anti-pattern was brought to my attention by a coworker, thanks.

To make a download, installation and setup of a tool more convenient, some vendors opt to distribute installation scripts via HTTP API endpoint. It is expected that user will download the script using curl and pipe it to a shell for execution:

~$ curl | bash

If it’s not obvious, this can cause security problems if vendor’s infrastructure is compromised. Optionally, if vendor changes the script and unwillingly introduces security misconfiguration, it will be propagated to your infrastructure.

When using such scripts for installation and configuration, do the following:

  1. Download the script manually and inspect it’s contents. Make sure you understand everything that script does.
  2. Calculate sha256sum of the script (or some other hash, but not SHA1 or MD5).
  3. Use the calculated hash in the subsequent runs to ensure contents of the script didn’t change.

You automation script then might look something like this:

~$ curl -O
~$ if [[ $(sha256sum get-tool-script | grep <sha256sum-that-you-calculated-earlier>) ]]; then \
     bash get-tool-script; \
   else \
     echo "Invalid SHA256 sum. Please check the script contents!"; \
Miroslav Bagľaš
Senior Security Engineer