Atlas Testing

dataset csv

dataset.csv attributes

Name and description	Required	Value
batch Maximum number of rows written per statement. Defaults to 1000.	false	int
exclude Skip the dataset tables matching the glob patterns.	false	List of strings
include Load only the dataset tables matching the glob patterns (orders, public.*).	false	List of strings
sampler Load a sample of each table, taken by a top-level sampler block, instead of all its rows.	false	Object reference to sampler
src The source of the snapshot: a directory, or an object-store URL (s3://, gs://, azblob://). Local paths are relative to the test file.	true	string

dataset rds_snapshot

dataset.rds_snapshot attributes

Name and description	Required	Value
batch Maximum number of rows written per statement. Defaults to 1000.	false	int
exclude Skip the dataset tables matching the glob patterns.	false	List of strings
include Load only the dataset tables matching the glob patterns (orders, public.*).	false	List of strings
sampler Load a sample of each table, taken by a top-level sampler block, instead of all its rows.	false	Object reference to sampler
src The source of the snapshot: a directory, or an object-store URL (s3://, gs://, azblob://). Local paths are relative to the test file.	true	string

sampler composite

Sample each table by the sampler of the first table block matching it, and load all rows of the tables none matches.

sampler "composite" "prod" {
  table {
    match = ["events", "clicks", "audit_*"]
    use   = sampler.random.large
  }
  table {
    match = ["countries", "currencies"]
    all   = true
  }
  table {
    match = ["*"]
    use   = sampler.head.small
  }
}

sampler.composite blocks

sampler.table

Sample the tables matching the glob patterns, unless a table block before it matches them.

sampler.table attributes

Name and description	Required	Value
all Set to true to load all rows of the matching tables, in place of use.	false	bool
match The glob patterns matching the tables, as include does (orders, public.audit_*).	true	List of strings
use The head, tail or random sampler taking the sample of the matching tables.	false	Object reference to sampler

sampler.table constraints

Constraint	Value
Required	true
Require Name	false
Repeatable	true
Mutually exclusive sets	[use, all]
One of required sets	[use, all]

sampler head

Take the first rows of each table, in the snapshot order.

sampler.head attributes

Name and description	Required	Value
related Also load the rows referencing the sampled ones, in the same table or another loaded one: one level down, as these rows load the rows they reference, but not the rows referencing them. The tables referencing a related sample are read whole.	false	bool
rows The number of rows each table takes, or all its rows if it has fewer. The rows these reference, in the same table or another loaded one, are loaded on top, so no foreign key is left dangling.	true	int

sampler random

Take rows drawn uniformly at random, the same ones on every run over the same snapshot, and a larger sample keeps the rows of a smaller one. The rows of each table are counted first.

sampler.random attributes

Name and description	Required	Value
related Also load the rows referencing the sampled ones, in the same table or another loaded one: one level down, as these rows load the rows they reference, but not the rows referencing them. The tables referencing a related sample are read whole.	false	bool
rows The number of rows each table takes, or all its rows if it has fewer. The rows these reference, in the same table or another loaded one, are loaded on top, so no foreign key is left dangling.	true	int
seed Selects another random draw. Defaults to 0.	false	int

sampler tail

Take the last rows of each table, in the snapshot order. The rows of each table are counted first.

sampler.tail attributes

Name and description	Required	Value
related Also load the rows referencing the sampled ones, in the same table or another loaded one: one level down, as these rows load the rows they reference, but not the rows referencing them. The tables referencing a related sample are read whole.	false	bool
rows The number of rows each table takes, or all its rows if it has fewer. The rows these reference, in the same table or another loaded one, are loaded on top, so no foreign key is left dangling.	true	int

test migrate

The test "migrate" "name" block describes a migrate test case.

test "migrate" "20240613061102" {
  migrate {
    to = "20240613061046"
  }
  exec {
    sql = "INSERT INTO users (name) VALUES ('Ada Lovelace')"
  }
  migrate {
    to = "20240613061102"
  }
  exec {
    sql = "SELECT first_name,last_name FROM users"
    output = "Ada, Lovelace"
  }
}

test.migrate attributes

Name	Required	Value
skip	false	bool

test.migrate blocks

test.assert

The assert block executes an SQL statement and expects a single-row, single-column result with a true value.

test.assert attributes

Name	Required	Value
error_message	false	string
sql	true	string

test.assert constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.catch

The catch block executes one or more SQL statements and expects an error to occur.

test.catch attributes

Name	Required	Value
error	false	string
sql	true	string

test.catch constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.cleanup

The cleanup block registers a command to be called when the test case is completed.

cleanup {
  sql = "SELECT pg_catalog.set_config('search_path', '', false)"
}

test.cleanup attributes

Name	Required	Value
sql	true	string

test.cleanup constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.dataset

The dataset block loads a dataset registered by a top-level dataset block into the test database, at the point of the test case it appears in.

dataset {
  use = dataset.rds_snapshot.prod
}

test.dataset attributes

Name	Required	Value
use	true	Object reference to dataset

test.dataset constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.exec

The exec block executes one or more SQL statements and expects all of them to succeed. If output or match is defined, the output of the SQL statement is compared to the expected value.

test.exec attributes

Name	Required	Value
format	false	enum (csv, table)
match	false	string
output	false	string
sql	true	string

test.exec constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.external

The external allows executing external programs to insert data or run assertions on the dev-database used by test case.

external {
  program = [
    "go", "run", "seed.go",
    "--dev-url", testing.dev_url,
  ]
}

test.external attributes

Name	Required	Value
match	false	string
output	false	string
program	true	List of strings
working_dir	false	string

test.external constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.log

The log block logs a message to the test output.

test.log attributes

Name	Required	Value
message	true	string

test.log constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.migrate

test.migrate attributes

Name	Required	Value
to	true	string

test.migrate constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.script.exec

test.script.exec attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.exec blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.exec constraints

Constraint	Value
Required	false
Repeatable	true

test.script.loop

test.script.loop attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.loop blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.loop constraints

Constraint	Value
Required	false
Repeatable	true

test.script.query

test.script.query attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.query blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.query constraints

Constraint	Value
Required	false
Repeatable	true

test plan

The test "plan" "name" block describes a schema plan test case.

test "plan" "20240613061102" {
  schema {
    url = "atlas://repo?version=20240613061046"
  }
  exec {
    sql = "INSERT INTO users (name) VALUES ('Ada Lovelace')"
  }
  apply {
    url = "file://name.plan.hcl"
  }
  exec {
    sql = "SELECT first_name,last_name FROM users"
    output = "Ada, Lovelace"
  }
}

test.plan attributes

Name	Required	Value
skip	false	bool

test.plan blocks

test.apply

test.apply attributes

Name	Required	Value
url	true	string

test.apply constraints

Constraint	Value
Required	true
Require Name	false

test.assert

The assert block executes an SQL statement and expects a single-row, single-column result with a true value.

test.assert attributes

Name	Required	Value
error_message	false	string
sql	true	string

test.assert constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.catch

The catch block executes one or more SQL statements and expects an error to occur.

test.catch attributes

Name	Required	Value
error	false	string
sql	true	string

test.catch constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.cleanup

The cleanup block registers a command to be called when the test case is completed.

cleanup {
  sql = "SELECT pg_catalog.set_config('search_path', '', false)"
}

test.cleanup attributes

Name	Required	Value
sql	true	string

test.cleanup constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.dataset

The dataset block loads a dataset registered by a top-level dataset block into the test database, at the point of the test case it appears in.

dataset {
  use = dataset.rds_snapshot.prod
}

test.dataset attributes

Name	Required	Value
use	true	Object reference to dataset

test.dataset constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.exec

The exec block executes one or more SQL statements and expects all of them to succeed. If output or match is defined, the output of the SQL statement is compared to the expected value.

test.exec attributes

Name	Required	Value
format	false	enum (csv, table)
match	false	string
output	false	string
sql	true	string

test.exec constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.external

The external allows executing external programs to insert data or run assertions on the dev-database used by test case.

external {
  program = [
    "go", "run", "seed.go",
    "--dev-url", testing.dev_url,
  ]
}

test.external attributes

Name	Required	Value
match	false	string
output	false	string
program	true	List of strings
working_dir	false	string

test.external constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.log

The log block logs a message to the test output.

test.log attributes

Name	Required	Value
message	true	string

test.log constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.schema

test.schema attributes

Name	Required	Value
url	true	string

test.schema constraints

Constraint	Value
Required	true
Require Name	false

test.script.exec

test.script.exec attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.exec blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.exec constraints

Constraint	Value
Required	false
Repeatable	true

test.script.loop

test.script.loop attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.loop blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.loop constraints

Constraint	Value
Required	false
Repeatable	true

test.script.query

test.script.query attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.query blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.query constraints

Constraint	Value
Required	false
Repeatable	true

test schema

The test "schema" "name" block describes a schema test case.

test "schema" "postal" {
  parallel = true
  exec {
    sql = "SELECT '12345'::us_postal_code"
  }
  catch {
    sql = "SELECT 'hello'::us_postal_code"
  }
}

test.schema attributes

Name	Required	Value
parallel	false	bool
skip	false	bool

test.schema blocks

test.assert

The assert block executes an SQL statement and expects a single-row, single-column result with a true value.

test.assert attributes

Name	Required	Value
error_message	false	string
sql	true	string

test.assert constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.catch

The catch block executes one or more SQL statements and expects an error to occur.

test.catch attributes

Name	Required	Value
error	false	string
sql	true	string

test.catch constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.cleanup

The cleanup block registers a command to be called when the test case is completed.

cleanup {
  sql = "SELECT pg_catalog.set_config('search_path', '', false)"
}

test.cleanup attributes

Name	Required	Value
sql	true	string

test.cleanup constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.dataset

The dataset block loads a dataset registered by a top-level dataset block into the test database, at the point of the test case it appears in.

dataset {
  use = dataset.rds_snapshot.prod
}

test.dataset attributes

Name	Required	Value
use	true	Object reference to dataset

test.dataset constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.exec

The exec block executes one or more SQL statements and expects all of them to succeed. If output or match is defined, the output of the SQL statement is compared to the expected value.

test.exec attributes

Name	Required	Value
format	false	enum (csv, table)
match	false	string
output	false	string
sql	true	string

test.exec constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.external

The external allows executing external programs to insert data or run assertions on the dev-database used by test case.

external {
  program = [
    "go", "run", "seed.go",
    "--dev-url", testing.dev_url,
  ]
}

test.external attributes

Name	Required	Value
match	false	string
output	false	string
program	true	List of strings
working_dir	false	string

test.external constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.log

The log block logs a message to the test output.

test.log attributes

Name	Required	Value
message	true	string

test.log constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.script.exec

test.script.exec attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.exec blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.exec constraints

Constraint	Value
Required	false
Repeatable	true

test.script.loop

test.script.loop attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.loop blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.loop constraints

Constraint	Value
Required	false
Repeatable	true

test.script.query

test.script.query attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.query blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.query constraints

Constraint	Value
Required	false
Repeatable	true

test script

The test "script" "name" block describes a script test case. The optional schema block sets up the schema the script expects and must be defined before any other block.

test "script" "vip_backfill" {
  schema {
    url = "file://schema.hcl"
  }
  exec {
    sql = "INSERT INTO users (id, vip, tier) VALUES (1, true, NULL)"
  }
  script "exec" {
    file = "vip_backfill.script.hcl"
  }
  assert {
    sql = "SELECT count(*) = 0 FROM users WHERE vip AND tier IS NULL"
  }
}

test.script attributes

Name	Required	Value
skip	false	bool

test.script blocks

test.assert

The assert block executes an SQL statement and expects a single-row, single-column result with a true value.

test.assert attributes

Name	Required	Value
error_message	false	string
sql	true	string

test.assert constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.catch

The catch block executes one or more SQL statements and expects an error to occur.

test.catch attributes

Name	Required	Value
error	false	string
sql	true	string

test.catch constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.cleanup

The cleanup block registers a command to be called when the test case is completed.

cleanup {
  sql = "SELECT pg_catalog.set_config('search_path', '', false)"
}

test.cleanup attributes

Name	Required	Value
sql	true	string

test.cleanup constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.dataset

The dataset block loads a dataset registered by a top-level dataset block into the test database, at the point of the test case it appears in.

dataset {
  use = dataset.rds_snapshot.prod
}

test.dataset attributes

Name	Required	Value
use	true	Object reference to dataset

test.dataset constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.exec

The exec block executes one or more SQL statements and expects all of them to succeed. If output or match is defined, the output of the SQL statement is compared to the expected value.

test.exec attributes

Name	Required	Value
format	false	enum (csv, table)
match	false	string
output	false	string
sql	true	string

test.exec constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.external

The external allows executing external programs to insert data or run assertions on the dev-database used by test case.

external {
  program = [
    "go", "run", "seed.go",
    "--dev-url", testing.dev_url,
  ]
}

test.external attributes

Name	Required	Value
match	false	string
output	false	string
program	true	List of strings
working_dir	false	string

test.external constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true
Mutually exclusive sets	[output, match]

test.log

The log block logs a message to the test output.

test.log attributes

Name	Required	Value
message	true	string

test.log constraints

Constraint	Value
Required	false
Require Name	false
Repeatable	true

test.schema

test.schema attributes

Name	Required	Value
url	true	string

test.script.exec

test.script.exec attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.exec blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.exec constraints

Constraint	Value
Required	false
Repeatable	true

test.script.loop

test.script.loop attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.loop blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.loop constraints

Constraint	Value
Required	false
Repeatable	true

test.script.query

test.script.query attributes

Name	Required	Value
error	false	string
file	true	string
match	false	string
output	false	string
run	false	string
vars	false	map

test.script.query blocks

test.script.as

test.script.as attributes

Name	Required	Value
password	false	string
role	false	string
user	false	string

test.script.as constraints

Constraint	Value
Required	false
Require Name	false
Mutually exclusive sets	[role, user]
One of required sets	[role, user]

test.script.query constraints

Constraint	Value
Required	false
Repeatable	true