Manage Environment Variables with Pydantic

Introduction

Builders work on purposes which are speculated to be deployed on some server to be able to permit anybody to make use of these. Sometimes within the machine the place these apps dwell, builders arrange atmosphere variables that permit the app to run. These variables might be API keys of exterior companies, URL of your database and rather more.

For native improvement although, it’s actually inconvenient to declare these variables on the machine as a result of it’s a gradual and messy course of. So I’d prefer to share on this quick tutorial the best way to use Pydantic to deal with atmosphere variables in a safe manner.

.env file

What you generally do in a Python challenge is to retailer all of your atmosphere variables in a file named .env. It is a textual content file containing all of the variables in a key : worth format. You need to use additionally the worth of one of many variables to declare one of many different variables by leveraging the {} syntax.

The next is an instance:

#.env file

OPENAI_API_KEY="sk-your non-public key"
OPENAI_MODEL_ID="gpt-4o-mini"

# Growth settings
DOMAIN=instance.org
ADMIN_EMAIL=admin@${DOMAIN}

WANDB_API_KEY="your-private-key"
WANDB_PROJECT="myproject"
WANDB_ENTITY="my-entity"

SERPAPI_KEY= "your-api-key"
PERPLEXITY_TOKEN = "your-api-token"

Bear in mind the .env file ought to stay non-public, so it can be crucial that this file is talked about in your .gitignore file, to ensure that you by no means push it on GitHub, in any other case, different builders might steal your keys and use the instruments you’ve paid for.

env.instance file

To ease the lifetime of builders who will clone your repository, you may embody an env.instance file in your challenge. It is a file containing solely the keys of what’s supposed to enter the .env file. On this manner, different individuals know what APIs, tokens, or secrets and techniques usually they should set to make the scripts work.

#env.instance

OPENAI_API_KEY=""
OPENAI_MODEL_ID=""

DOMAIN=""
ADMIN_EMAIL=""

WANDB_API_KEY=""
WANDB_PROJECT=""
WANDB_ENTITY=""

SERPAPI_KEY= ""
PERPLEXITY_TOKEN = ""

python-dotenv is the library you utilize to load the variables declared into the .env file. To put in this library:

pip set up python-dotenv

Now you should use the load_dotenv to load the variables. Then get a reference to those variables with the os module.

import os
from dotenv import load_dotenv

load_dotenv()

OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
OPENAI_MODEL_ID = os.getenv('OPENAI_MODEL_ID')

This technique will first look into your .env file to load the variables you’ve declared there. If this file doesn’t exist, the variable will probably be taken from the host machine. Because of this you should use the .env file to your native improvement however then when the code is deployed to a bunch atmosphere like a digital machine or Docker container we’re going to straight use the atmosphere variables outlined within the host atmosphere.

Pydantic

Pydantic is without doubt one of the most used libraries in Python for knowledge validation. It’s also used for serializing and deserializing courses into JSON and again. It robotically generates JSON schema, decreasing the necessity for guide schema administration. It additionally gives built-in knowledge validation, guaranteeing that the serialized knowledge adheres to the anticipated format. Lastly, it simply integrates with well-liked net frameworks like FastAPI.

pydantic-settings is a Pydantic characteristic wanted to load and validate settings or config courses from atmosphere variables.

!pip set up pydantic-settings

We’re going to create a category named Settings. This class will inherit BaseSettings. This makes the default behaviours of figuring out the values of any fields to be learn from the .env file. If no var is discovered within the .env file will probably be used the default worth if supplied.

from pydantic_settings import BaseSettings, SettingsConfigDict

from pydantic import (
    AliasChoices,
    Area,
    RedisDsn,
)


class Settings(BaseSettings):
    auth_key: str = Area(validation_alias="my_auth_key")  
    api_key: str = Area(alias="my_api_key")  

    redis_dsn: RedisDsn = Area(
        'redis://person:cross@localhost:6379/1', #default worth
        validation_alias=AliasChoices('service_redis_dsn', 'redis_url'),  
    )

    model_config = SettingsConfigDict(env_prefix='my_prefix_')

Within the Settings class above now we have outlined a number of fields. The Area class is used to offer extra information about an attribute.

In our case, we setup a validation_alias. So the variable identify to search for within the .env file is overridden. Within the case reported above, the atmosphere variable my_auth_key will probably be learn as an alternative of auth_key.

You can too have a number of aliases to search for within the .env file you could specify by leveraging AliasChoises(choise1, choise2).

The final attribute model_config , comprises all of the variables concerning a specific matter (e.g connection to a db). And this variable will retailer all .env var that begin with the prefix env_prefix.

Instantiate and use settings

The following step can be to really instantiate and use these settings in your Python challenge.

from pydantic_settings import BaseSettings, SettingsConfigDict

from pydantic import (
    AliasChoices,
    Area,
    RedisDsn,
)


class Settings(BaseSettings):
    auth_key: str = Area(validation_alias="my_auth_key")  
    api_key: str = Area(alias="my_api_key")  

    redis_dsn: RedisDsn = Area(
        'redis://person:cross@localhost:6379/1', #default worth
        validation_alias=AliasChoices('service_redis_dsn', 'redis_url'),  
    )

    model_config = SettingsConfigDict(env_prefix='my_prefix_')

# create instantly a settings object
settings = Settings()

Now what use the settings in different components of our codebase.

from Settings import settings

print(settings.auth_key) 

You lastly have an easy accessibility to your settings, and Pydantic helps you validate that the secrets and techniques have the right format. For extra superior validation ideas seek advice from the Pydantic documentation: https://docs.pydantic.dev/latest/

Ultimate ideas

Managing the configuration of a challenge is a boring however essential a part of software program improvement. Secrets and techniques like API keys, db connections are what often energy your software. Naively you possibly can hardcode these variables in your code and it’ll nonetheless work, however for apparent causes, this might not be an excellent observe. On this article, I confirmed you an introduction on the best way to use pydantic settings to have a structured and secure solution to deal with your configurations.

💼 Linkedin ️| 🐦 X (Twitter) | 💻 Website