Secure Your Docker Stack: A Comprehensive Guide to Docker Compose Secrets

Dragos
September 6, 2024
- Vps

Docker Compose is a powerful tool for defining and running multi-container Docker applications. It allows developers to use a YAML file to configure application services, networks, and volumes, simplifying the process of managing complex containerized environments. However, as applications grow in complexity and sensitivity, managing confidential information such as API keys, passwords, and other secrets becomes increasingly challenging.

In today’s security-conscious world, protecting sensitive data is paramount. Hardcoding secrets directly into Dockerfiles or Docker Compose files is a risky practice that can lead to security breaches if these files are exposed. This is where Docker secrets come into play, offering a secure way to manage sensitive information in containerized environments.

This article will guide you through the process of using secrets in Docker Compose, helping you enhance the security of your Docker-based applications without compromising on convenience or functionality.

Understanding Docker Secrets

What are Docker Secrets?

Docker secrets are a feature provided by Docker to manage sensitive data securely. A secret is a blob of data, such as a password, SSH private key, SSL certificate, or any other piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code.

Docker secrets are:

Encrypted at rest
Encrypted during transit
Mounted as files in the container
Centrally managed by Docker Swarm (required for Docker secrets)

Benefits of using secrets in Docker Compose

Enhanced Security: Secrets are encrypted and only accessible to services that explicitly need them, reducing the risk of exposure.
Centralized Management: All secrets are managed in one place, making it easier to update, rotate, or revoke them as needed.
Separation of Concerns: Developers can work with Docker Compose files without needing access to the actual secret values.
Version Control Friendly: Since secrets are not stored in the Docker Compose file itself, you can safely version control your configuration without exposing sensitive data.
Runtime Injection: Secrets are injected into containers at runtime, not build time, allowing for more flexible and secure deployments.
Scalability: As your application scales, Docker secrets provide a consistent way to manage sensitive data across multiple containers and nodes.

By leveraging Docker secrets in your Docker Compose setup, you can significantly improve the security posture of your containerized applications while maintaining the ease of use that Docker Compose provides.

Setting Up Docker Secrets

Setting up Docker secrets involves two main steps: creating the secret files and using the Docker CLI to add these secrets to your Docker environment.

Creating secret files

Create a file containing your secret information. For example:
```
echo "mysupersecretpassword" > db_password.txt
```
Ensure the file permissions are restricted:
```
chmod 400 db_password.txt
```

Using the `docker secret create` command

Once you have your secret file, you can create a Docker secret using the following command:

docker secret create my_db_password db_password.txt

This command does the following:

docker secret create is the command to create a new secret
my_db_password is the name you’re giving to this secret in Docker
db_password.txt is the file containing the secret

You can verify the secret was created by running:

docker secret ls

Common Commands

Command	Description
`docker secret create`	Creates a new secret
`docker secret ls`	Lists all secrets
`docker secret inspect`	Shows details of a specific secret
`docker secret rm <secret>`	Removes a secret

Remember, after creating the secret, you can safely delete the local file as Docker now securely stores the secret.

Implementing Secrets in Docker Compose

Add the Docker External Secret Created Previousely

To use secrets in your Docker Compose setup, you need to declare them in your docker-compose.yml file and then configure your services to access these secrets. This will use the secret that you have defined initialy external.

Syntax for declaring secrets in docker-compose.yml

Here’s an example of how to declare and use secrets in a Docker Compose file:

version: '3.8'

services:
  myapp:
    image: myapp:latest
    secrets:
      - my_db_password
    environment:
      - DB_PASSWORD_FILE=/run/secrets/my_db_password

secrets:
  my_db_password:
    external: true

Let’s break this down:

The secrets section at the root level declares which secrets this compose file uses.
external: true indicates that this secret has been created externally (using docker secret create).
In the services section, we list the secrets that should be made available to the service.
The environment section shows how to reference the secret file path

Accessing secrets in containers

Docker mounts secrets as files under /run/secrets/<secret_name> in the container. In the example above, the secret would be accessible at /run/secrets/my_db_password.

To use the secret in your application, you would read the contents of this file. Here’s an example in Python:

with open('/run/secrets/my_db_password', 'r') as secret_file:
    db_password = secret_file.read().strip()

# Now use db_password to connect to your database

Remember, it’s the responsibility of your application code to read the secret from the file and use it appropriately.

Integrating Secrets Directly into Docker Compose Files

This section uses directly the docker compose file to define the secret you don’t need to create it before:

Step 1: Define Secrets

First, define the secrets in your docker-compose.yml file.

version: '3.7'
secrets:
  my_secret:
    file: ./my_secret.txt

Step 2: Use Secrets in Services

Next, reference the secrets in your service definitions.

services:
  my_service:
    image: my_image
    secrets:
      - my_secret

Step 3: Access Secrets in Containers

The secrets will be available in your container at /run/secrets.

cat /run/secrets/my_secret

Example Configuration

Here’s a complete example:

version: '3.7'
secrets:
  db_password:
    file: ./db_password.txt
services:
  database:
    image: mysql:latest
    environment:
      MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

A fule exemple confiuration can be checked for traefik setup with wildcard certificat in here you have a full docker compose file will the things.

Differences Between External and Internal Docker Compose Secrets

Docker Compose allows you to work with two types of secrets: external and internal. Understanding the differences between these types is crucial for effective secret management.

External Secrets

External secrets are created and managed outside of the Docker Compose file, typically using the Docker CLI or a secret management system.

Characteristics:

Created independently of the Docker Compose file
Referenced in the Compose file using the external: true attribute
Provide better separation of concerns and enhanced security
Can be shared across multiple services and stacks

Internal Secrets

Internal secrets are defined directly within the Docker Compose file.

Characteristics:

Defined and managed within the Docker Compose file
Can be created from files, environment variables, or directly in the Compose file
Easier to set up for development and testing environments
Less secure as they may be visible in the Compose file

Key Differences

Management: External secrets are managed outside the Compose file, while internal secrets are defined within it.
Security: External secrets offer better security as they’re not visible in the Compose file.
Reusability: External secrets can be easily shared across multiple services and stacks.
Deployment: Internal secrets are easier to deploy in development environments but may require additional steps in production.
Versioning: External secrets can be versioned independently of the application code.

When to Use Each Type

Use external secrets for:
- Production environments
- Sensitive data that needs to be shared across multiple services
- When you need to manage secrets independently of your application code
Use internal secrets for:
- Development and testing environments
- Quick prototyping
- When the secret is specific to a single service and doesn’t need to be shared

By understanding these differences, you can choose the most appropriate method for managing secrets in your Docker Compose projects, balancing security needs with ease of use and deployment considerations.

Best Practices

When working with Docker secrets, following best practices ensures optimal security and manageability. Here are some key recommendations:

Naming conventions

Use descriptive names: Choose secret names that clearly indicate their purpose, e.g., prod_db_password instead of secret1.
Use consistent prefixes: For example, use prod_ for production secrets and dev_ for development secrets.
Avoid including sensitive information in names: The secret name itself shouldn’t reveal any confidential data.

Example:

secrets:
  prod_api_key:
    external: true
  dev_db_password:
    external: true

Version control considerations

Never commit actual secret values to version control.
Use .gitignore to exclude files containing secrets.
Consider using a .env.example file to show which environment variables are needed, without including actual values.

Example .gitignore entry:

*.env
secrets/

Rotating secrets

Regularly update secrets to minimize the impact of potential breaches.
Use Docker’s secret rotation feature in Swarm mode:
1. Create a new secret with a versioned name.
2. Update your service to use the new secret.
3. Remove the old secret after ensuring all services are updated.

Example:

docker secret create db_password_v2 new_password.txt
docker service update --secret-rm db_password_v1 --secret-add db_password_v2 myservice
docker secret rm db_password_v1

Additional best practices

Limit secret access: Only give services access to the secrets they absolutely need.
Use secret drivers: Consider using external secret management systems through Docker secret drivers for advanced use cases.
Audit secret usage: Regularly review which services are using which secrets and revoke unnecessary access.

Example Use Cases

Docker secrets can be used in various scenarios to enhance the security of your containerized applications. Here are some common use cases:

Database credentials

Securely manage database passwords without hardcoding them in your application or Docker files.

version: '3.8'
services:
  db:
    image: postgres
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

  app:
    image: myapp
    secrets:
      - db_password
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    external: true

API keys

Safely use API keys in your services without exposing them in your code or configuration files.

version: '3.8'
services:
  api_service:
    image: api_service
    secrets:
      - api_key
    environment:
      API_KEY_FILE: /run/secrets/api_key

secrets:
  api_key:
    external: true

SSL certificates

Manage SSL certificates securely for services that require HTTPS.

version: '3.8'
services:
  web:
    image: nginx
    secrets:
      - site_certificate
      - site_key
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro

secrets:
  site_certificate:
    file: ./certs/site.crt
  site_key:
    file: ./certs/site.key

JWT signing keys

Securely manage keys used for signing JSON Web Tokens (JWTs) in authentication services.

version: '3.8'
services:
  auth_service:
    image: auth_service
    secrets:
      - jwt_private_key
      - jwt_public_key
    environment:
      JWT_PRIVATE_KEY_FILE: /run/secrets/jwt_private_key
      JWT_PUBLIC_KEY_FILE: /run/secrets/jwt_public_key

secrets:
  jwt_private_key:
    external: true
  jwt_public_key:
    external: true

By leveraging Docker secrets in these scenarios, you can significantly enhance the security of your containerized applications while maintaining flexibility and ease of management.

Limitations and Alternatives

While Docker secrets provide a robust solution for managing sensitive information in containerized environments, it’s important to be aware of their limitations and consider alternatives when necessary.

Swarm mode requirement

The primary limitation of Docker secrets is that they require Docker Swarm mode to be enabled. This can be a significant drawback for users who prefer to use Docker in standalone mode or with other orchestration tools like Kubernetes.

Implications:

Not suitable for single-node deployments without Swarm
May require architectural changes in existing non-Swarm setups

Other options for managing sensitive data

Given the limitations of Docker secrets, here are some alternatives to consider:

Environment Variables:
- Pros: Simple to use, widely supported
- Cons: Can be exposed through system calls, not encrypted at rest
Example:
```
services:
  app:
    image: myapp
    environment:
      - DB_PASSWORD=mysecretpassword
```
Config Management Tools:
- Options: Ansible, Puppet, Chef
- Pros: Powerful, flexible, can manage configurations across multiple servers
- Cons: Require additional setup and knowledge
Cloud Provider Secret Management Services:
- Options: AWS Secrets Manager, Google Cloud Secret Manager, Azure Key Vault
- Pros: Highly secure, integrated with cloud ecosystems
- Cons: Vendor lock-in, potential cost implications

HashiCorp Vault:

Pros: Highly secure, platform-agnostic, supports dynamic secrets
Cons: Complex setup, requires additional infrastructure

Example integration:

services:
  app:
    image: myapp
    environment:
      - VAULT_ADDR=http://vault:8200
    entrypoint: ["vault-agent", "-config=/vault-agent-config.hcl"]

Docker Config:
- Similar to Docker secrets but for non-sensitive configuration data
- Can be used in conjunction with secrets for a comprehensive configuration management strategy

When choosing an alternative, consider factors such as your specific security requirements, existing infrastructure, team expertise, and scalability needs.

Conclusions

Using secrets in Docker Compose adds a vital layer of security to your applications. By keeping sensitive information out of your codebase, you reduce the risk of leaks and unauthorized access.

Start implementing secrets today to bolster your application’s security. With these steps, you’ll handle confidential data more responsibly and effectively, ensuring a safer deployment environment.

#Docker