In software development processes, applications require sensitive data (secrets) such as API keys, database credentials, passwords, and certificates to function properly. However, storing these secrets within code, configuration files, or environment variables carries significant security risks. Leaks, unauthorized access, or misconfigurations can lead to major data breaches.

Why Traditional Methods Fall Short?

• Environment Variables: Often visible in process lists, difficult to manage in distributed systems.
• Configuration Files: Can be accidentally committed to source control systems (like Git), open to unauthorized access.
• Hard-coding into Code: One of the worst practices, requiring code recompilation if secrets need to be changed or updated.

To address these issues, we need a centralized secrets management solution like HashiCorp Vault. Vault is designed to securely store, access, and manage all the secrets your applications need.

What is HashiCorp Vault?

HashiCorp Vault is a tool that provides centralized storage of secrets, access control, and encryption services. It enables developers to securely access sensitive information used in their applications. Key features of Vault include:

• Secrets Storage: Can store Key/Value pairs, dynamic secrets (database credentials), certificates, etc.
• Data Encryption: Provides an API for data encryption/decryption operations, offering encryption services without requiring the data itself to be stored in Vault.
• Access Control: Defines who can access which secret for how long with detailed policies (ACL).
• Audit Log: Audits all secret access and management activities.

Basic Vault Setup and Usage for Developers (Local Environment)

Starting Vault in dev mode in your local development environment is quite easy. This mode is for testing and learning purposes; a different configuration is required for production environments.

vault server -dev -dev-listen-address="0.0.0.0:8200"

This command starts Vault in development mode and by default shows the root token. You can log into Vault using this token:

export VAULT_ADDR='http://127.0.0.1:8200' # Default 127.0.0.1:8200 if dev-listen-address was not used.

vault login <root_token>

Using the KV Secrets Engine (Key/Value Store)

One of the most commonly used secrets engines is the KV (Key/Value) engine. It is ideal for storing static secrets like application configurations and API keys. The KV-v2 engine provides versioning support.

First, let's enable the KV-v2 engine:

vault secrets enable -version=2 kv

Now let's create a secret. For example, an API key and database username for our application:

vault kv put kv/myapp/config api_key="superSecretAPIKey123" db_user="app_admin" db_password="Pa$$w0rd"

To read this secret:

vault kv get kv/myapp/config

--- Key --- Value ---version     1api_key     superSecretAPIKey123db_password Pa$$w0rddb_user     app_admin

Fetching Secrets with a Python Application

Now let's see how to fetch these secrets via a Python application. We will use the hvac library.

pip install hvac

import hvacimport os# Vault server addressVAULT_ADDR = os.getenv('VAULT_ADDR', 'http://127.0.0.1:8200')# Using root token in dev mode. More secure methods should be preferred in production.VAULT_TOKEN = os.getenv('VAULT_TOKEN', '<your_root_token_here>') # Replace with your root tokenclient = hvac.Client(url=VAULT_ADDR, token=VAULT_TOKEN)if client.is_authenticated():    print("Successfully connected to Vault!")    try:        read_response = client.secrets.kv.v2.read_secret_version(            mount_point='kv',            path='myapp/config'        )        api_key = read_response['data']['data']['api_key']        db_user = read_response['data']['data']['db_user']        db_password = read_response['data']['data']['db_password']        print(f"API Key: {api_key}")        print(f"DB User: {db_user}")        print(f"DB Password: {db_password}")    except Exception as e:        print(f"Error reading secret: {e}")else:    print("Vault authentication failed.")

Important Note: In production environments, instead of using a root_token, you should use one of the various authentication methods provided by Vault (e.g., AppRole, Kubernetes Service Account). This example demonstrates the basic working principle.

Policy Management (ACL)

Everything in Vault is based on policies (ACL - Access Control List). You define who can access which path with what permissions using policies. For example, let's create a policy that grants only read access for the myapp-config path.

Let's create a file named myapp-policy.hcl:

path "kv/data/myapp/config" {  capabilities = ["read"]}

Write this policy to Vault:

vault policy write myapp-readonly myapp-policy.hcl

Now, by associating this policy with a token or an authentication method (e.g., an AppRole), we can ensure that only users/applications authorized by this policy can read secrets from the kv/data/myapp/config path.

Dynamic Secrets: Database Credentials

One of Vault's most powerful features is dynamic secrets. With this feature, applications can receive credentials that are created on demand, short-lived, and valid only for that specific request. For example, we can create dynamic users for a database. This way, even if a secret is leaked, the risk is minimized due to its short lifespan.

Let's enable the dynamic secrets engine for PostgreSQL:

vault secrets enable postgresql

Configure database connection information (this example assumes a PostgreSQL running on Docker):

vault write postgresql/config/connection connection_url="postgresql://{{username}}:{{password}}@localhost:5432/mydb"

Now, let's define a role. This role will create a new database user for a specified duration (e.g., 1 minute):

vault write postgresql/roles/my-app-role 
oles="myapp_read_only" 	tl="1m" 
enewal_period="1m" 
ative_tls=false 
evoke_on_delete=true 
evocation_sql="DROP USER IF EXISTS "{{name}}" CASCADE;" \
creation_statements="CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}";"

Request dynamic credentials from this role:

vault read postgresql/creds/my-app-role

Key                Value---                -----lease_id           postgresql/creds/my-app-role/<UUID>lease_duration     1mlease_renewable    truedata              map[password:g9e...N92 username:v-token-my-app-role-<UUID>]

As you can see, Vault automatically generated a username and password, and these credentials are valid for only 1 minute. Your application connects to the database using these credentials, and when the time expires, Vault can automatically revoke or renew this user.

Conclusion

HashiCorp Vault is a powerful tool that makes secrets management in modern applications secure and scalable. By eliminating the security vulnerabilities of environment variables or configuration files, you can manage your secrets from a central location, enforce access policies, and even create dynamic, short-lived credentials. As software developers, we must integrate such tools into our SDLC (Software Development Life Cycle) to build more secure applications.