Skip to main content

Can I store migration files in S3? (CLI and Terraform examples)

With Atlas, we advocate for treating migration directories as deployment artifacts resulting from a structured build process. The preferred approach is to push migration directories to the Atlas Schema Registry. In addition to its role as a migration directory storage, the Schema Registry provides a tight integration with the Atlas CLI and the Atlas Cloud UI, allowing you to deploy migrations, visualize schemas over time, review deployment logs and errors, and more.

However, some users prefer to store their migration directories in S3, typically due to internal policies or requirements.

Schema as Code: Directory Structure for HCL Files

Question​

How do I configure Atlas to load the database schema from nested directories with HCL files?

 /project
├── main.hcl
├── extensions
└── schemas
└── public
├── public.hcl
├── tables
│ ├── pets.hcl
│ ├── posts.hcl
│ ├── profiles.hcl
│ └── user_groups.hcl
├── functions/
└── types/

Managing PostgreSQL Extensions in a Dedicated Migration Process

PostgreSQL extensions are additional modules that extend the functionality of the PostgreSQL database. Examples of popular extensions include PostGIS for geographic data handling, PGVector for vector similarity search, and pgcrypto for cryptographic functions.

Once installed, these extensions provide additional data types and objects that users' schemas can leverage. However, because extensions are managed at the database level (and can only be installed once per database rather than per schema), users may prefer handling these extensions separately from their primary application schema migrations.

For example, if you maintain multiple applications – each with its own schema(s) but several relying on the same PostgreSQL extension – determining which application should handle installation, upgrades, or removal can be challenging. To address this complexity, managing extensions through a dedicated migration process is beneficial. This separation ensures extension-related changes remain isolated.

This guide demonstrates how to use the --include flag (or env.include argument) provided by Atlas to limit migrations specifically to PostgreSQL extensions.