Security with Docker Secrets

The Challenge of Managing Secrets in Applications

Managing secrets and sensitive configurations is a critical challenge in any application, especially in open-source projects. From the beginning of NotionAiAssistant, we considered security a priority, and proper management of credentials and API tokens was essential.

graph LR
    subgraph "Docker Secrets Flow"
        S1["Secrets Files<br>Local/CI/Production"] --> S2["Docker Compose<br>References Secrets"]
        S2 --> S3["Mounted at<br>/run/secrets/"]
        S3 --> S4["Application<br>Reads from Filesystem"]
    end
    
    subgraph "Security Benefits"
        S4 --> V1["Not Visible<br>in Logs"]
        S4 --> V2["Not Exposed<br>in Variables"]
        S4 --> V3["Not Included<br>in Images"]
    end
    
    classDef source fill:#f9f,stroke:#333,stroke-width:2px
    classDef flow fill:#bbf,stroke:#333,stroke-width:2px
    classDef benefit fill:#bfb,stroke:#333,stroke-width:2px
    
    class S1 source
    class S2,S3,S4 flow
    class V1,V2,V3 benefit

Why Docker Secrets?

There are several secret management solutions available in the market, such as:

• HashiCorp Vault
• AWS Secrets Manager
• Google Secret Manager
• Kubernetes Secrets
• Encrypted environment variables
• .env files with tools like dotenv

However, we chose Docker Secrets for the following reasons:

1. Native Integration with Our Stack

Since we were already using Docker and Docker Compose for our development and production environments, Docker Secrets offered seamless integration without additional tools.

2. Simplicity and Low Learning Curve

Docker Secrets provides a straightforward and intuitive approach:

# Example usage in docker-compose.yml
services:
  app:
    image: notionaiassistant:latest
    secrets:
      - notion_api_key
      - database_password
      - openai_api_key

secrets:
  notion_api_key:
    file: ./secrets/notion_api_key.txt
  database_password:
    file: ./secrets/database_password.txt
  openai_api_key:
    file: ./secrets/openai_api_key.txt

3. Security Across Different Environments

Docker Secrets allowed us to implement a consistent strategy for different environments:

• Development: Secrets stored locally in files
• CI/CD: Secrets injected from environment variables
• Production: Secrets managed by Docker Swarm

4. Prevents Accidental Exposure

Docker Secrets significantly reduces the risk of accidental credential exposure:

• Not visible in logs or command outputs
• Not exposed through environment variables
• Not included in images or containers
• Mounted as temporary files in /run/secrets/

Implementation in NotionAiAssistant

Our Docker Secrets implementation follows a consistent pattern throughout the project:

1. Directory Structure

NotionAiAssistant/
├── secrets/
│   ├── .gitignore        # Ignores all files except templates
│   ├── notion_api_key.txt.template
│   ├── database_password.txt.template
│   └── ...

2. Accessing Secrets in Code

We implemented a helper function to access secrets consistently:

def get_secret(secret_name: str) -> str:
    """
    Gets a secret from Docker Secrets or falls back to environment variables.
    
    Args:
        secret_name: Name of the secret to retrieve
        
    Returns:
        Secret content as a string
    """
    # Default Docker Secrets path
    secret_path = f"/run/secrets/{secret_name}"
    
    # Check if the secret exists as a file
    if os.path.exists(secret_path):
        with open(secret_path, "r") as f:
            return f.read().strip()
    
    # Fallback to environment variable (local development or CI)
    env_var = os.environ.get(secret_name.upper())
    if env_var:
        return env_var
        
    raise ValueError(f"Secret {secret_name} not found")

3. Setup for New Contributors

We created a setup script to simplify initial configuration:

#!/bin/bash
# setup-secrets.sh

echo "Setting up secrets for development environment..."

mkdir -p secrets

for template in secrets/*.template; do
    secret_file="${template%.template}"
    
    if [ ! -f "$secret_file" ]; then
        echo "Creating $secret_file"
        echo "development_value" > "$secret_file"
        echo "⚠️  Remember to replace the default value in $secret_file with your actual credentials"
    fi
done

echo "✅ Setup complete!"

Practical Benefits Observed

Adopting Docker Secrets brought tangible benefits to the project:

1. Zero accidental exposure of credentials in commits or logs
2. Simplified onboarding process for new contributors
3. Consistency across development and production environments
4. Easier credential rotation without rebuilding images
5. Improved auditability of access to sensitive information

Comparison with Alternatives

Lessons Learned

Our experience with Docker Secrets taught us important lessons:

1. The simplest solution is often the best: We didn't need complex tools to solve basic problems
2. Native solutions reduce friction: Using what's already available in the ecosystem minimizes failure points
3. Clear documentation is essential: Especially for security-related aspects
4. Consistent patterns enhance security: A uniform approach simplifies auditing and maintenance

Conclusion

Choosing Docker Secrets demonstrated that, in many cases, the native tools of your ecosystem can offer elegant solutions to complex problems. Instead of adding more complexity with specialized tools, first evaluate what's already available in your current stack.

This approach aligns perfectly with our project philosophy: using simple, effective open-source technologies to create robust and accessible solutions.