How to Authenticate with cURL Using Basic Auth

cURL is a powerful and versatile command-line tool for making HTTP requests and transferring data. It supports a wide range of protocols and options, making it a go-to utility for developers to interact with web APIs. One common requirement when working with APIs is authentication – verifying your identity to access protected resources. A simple authentication mechanism built into the HTTP protocol is Basic Auth. In this guide, we‘ll take an in-depth look at how to use cURL to make requests with Basic Auth to authenticate to APIs.

Content Navigation show

Understanding Basic Authentication

Basic authentication is a simple challenge and response scheme included in the HTTP protocol. With Basic Auth, a client sends the username and password in the Authorization header when making a request to a server. The credentials are formatted as a base64-encoded string of username:password.

Here‘s what a raw HTTP request with Basic Auth looks like:

GET /api/protected-resource HTTP/1.1
Host: api.example.com
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

The Authorization header contains the word "Basic" followed by a space and then the base64 encoded string of the username and password joined by a colon (username:password).

When the server receives a request with Basic Auth credentials, it decodes the Authorization header, extracting the username and password. It then compares them against its internal database of valid users. If there‘s a match, the server fulfills the request and returns the protected resource. If the credentials are missing or invalid, the server responds with a 401 Unauthorized status, indicating authentication is required.

It‘s important to note that Basic Auth provides very minimal security. The username and password are sent in plaintext with every request, merely encoded with base64 which is trivially decoded. Anyone who intercepts the HTTP messages can easily extract the credentials. Basic Auth should only be used over a secure HTTPS connection to encrypt the sensitive data in transit. Even with HTTPS, Basic Auth may not be suitable for highly sensitive credentials, especially if the server stores the credentials in an insecure manner.

Making Authenticated Requests with cURL

With that background in mind, let‘s look at how to use cURL to send requests with Basic Auth. We‘ll be using the -u or --user option to pass the username and password to cURL.

Here‘s the basic syntax:

curl -u username:password URL

Or with the verbose --user flag:

curl --user username:password URL

Replace username and password with your actual credentials, and URL with the endpoint you want to access. cURL automatically encodes the credentials with base64 and includes them in the Authorization header.

For example, let‘s say we want to access a protected resource at https://api.example.com/secret that requires Basic Auth. The username is "AzureDiamond" and the password is "hunter2". Here‘s how we‘d make that request with cURL:

curl -u AzureDiamond:hunter2 https://api.example.com/secret

If the credentials are valid, the server will return the secret resource. If not, it will respond with a 401 Unauthorized error.

We can also use cURL‘s -v or --verbose flag to print out information about the request, including the headers:

curl -u AzureDiamond:hunter2 https://api.example.com/secret -v

In the output, we‘ll see the Authorization header cURL generates:

> GET /secret HTTP/1.1
> Host: api.example.com
> Authorization: Basic QXp1cmVEaWFtb25kOmh1bnRlcjI=

The base64-encoded credentials are QXp1cmVEaWFtb25kOmh1bnRlcjI=. If we decode that string, it reveals the username and password:

$ echo QXp1cmVEaWFtb25kOmh1bnRlcjI= | base64 --decode
AzureDiamond:hunter2

This demonstrates why Basic Auth is insecure over plaintext HTTP connections. It‘s trivial to extract the credentials from base64.

Providing Credentials in a Separate File

For convenience and to avoid having sensitive credentials in shell history, you can store them in a separate file in the format username:password and tell cURL to read from that file with -u using the @ syntax:

curl -u @creds.txt https://api.example.com/secret

Where the creds.txt file contains:

AzureDiamond:hunter2

Making POST Requests with Basic Auth

So far we‘ve looked at making authenticated GET requests with cURL, but the same approach works for other HTTP methods like POST. Just add the -X POST option to set the method and -d to pass the request body:

curl -u AzureDiamond:hunter2 -X POST -d ‘{"message": "hello"}‘ https://api.example.com/post

This will send a POST request with the given JSON body and Basic Auth credentials.

Securing Sensitive Credentials

When working with sensitive data like passwords and API keys, it‘s crucial to handle them securely. Avoid hardcoding credentials into scripts or version control. Instead, use environment variables or secure secret management systems.

For example, instead of having the username and password directly in the cURL command, retrieve them from environment variables:

curl -u "$USERNAME:$PASSWORD" https://api.example.com/secret

Then provide the values when running the script:

USERNAME=AzureDiamond PASSWORD=hunter2 ./auth_example.sh

This way, the sensitive credentials are not stored in the script itself. Make sure to avoid printing secrets to stdout/stderr or logging them.

When storing credentials on disk temporarily, ensure the file permissions are set appropriately so only authorized users can read them. Securely erase the files after use with a utility like shred.

Automating Authenticated Requests

Using cURL with Basic Auth can be easily integrated into shell scripts and CI/CD pipelines for automated, authenticated interactions with APIs.

For example, we can write a simple shell script to make multiple authenticated requests:

#!/bin/bash

USERNAME=AzureDiamond 
PASSWORD=hunter2
API_URL=https://api.example.com

curl -u "$USERNAME:$PASSWORD" "$API_URL/resource1"
curl -u "$USERNAME:$PASSWORD" "$API_URL/resource2"
curl -u "$USERNAME:$PASSWORD" "$API_URL/resource3"

This script sets the username, password, and base API URL as variables, then makes authenticated requests to several endpoints.

We can also use cURL in CI/CD pipelines to run API tests or perform authenticated deployments. For example, here‘s how we might use cURL with Basic Auth in a GitHub Actions workflow:

name: API Tests

on: [push]

jobs:

  test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2

    - name: Run API tests
      env:
        USERNAME: ${{ secrets.API_USERNAME }}  
        PASSWORD: ${{ secrets.API_PASSWORD }}
      run: |
        curl -u "$USERNAME:$PASSWORD" https://api.example.com/test1
        curl -u "$USERNAME:$PASSWORD" https://api.example.com/test2

This workflow uses GitHub‘s encrypted secrets feature to securely provide the API credentials as environment variables. The curl commands then reference those environment variables to authenticate.

cURL Alternatives for Basic Auth

While cURL is a popular choice for making HTTP requests with Basic Auth, there are many other tools and libraries that support authenticated requests. Here are a few cURL alternatives:

  • Postman – A GUI tool for building and testing APIs. Provides an easy way to configure Basic Auth.

  • HTTPie – A user-friendly command-line HTTP client with intuitive syntax. Supports Basic Auth with --auth username:password.

  • Axios – A promise-based HTTP client for JavaScript. Provides an auth option to configure Basic Auth.

  • Requests – A simple and powerful HTTP library for Python. Has built-in support for Basic Auth authentication.

These tools can be useful if you prefer a graphical interface over cURL‘s command-line approach or if you need to integrate Basic Auth into an application using one of the supported languages.

Conclusion

Basic authentication is a simple way to protect access to web resources. While it has its limitations in terms of security, Basic Auth can be appropriate for some internal or low-risk APIs. With cURL, we can easily test APIs that use Basic Auth or script authenticated interactions.

Remember to always use HTTPS to encrypt sensitive credentials in transit, and take care to securely handle and store secrets. We covered several best practices like using environment variables and secure secret management.

By understanding how to set the Authorization header and provide credentials with cURL‘s -u flag, you‘re equipped to work with a wide variety of APIs and services that use Basic Auth. As we saw, integrating cURL into shell scripts and CI/CD pipelines enables powerful automation workflows.

I hope this deep dive into using cURL with Basic Auth has been helpful! Let me know if you have any other questions.