Pulumi Testing & Validation

Testing Overview

Why Test Infrastructure Code?

The Problem: Infrastructure bugs are expensive. A misconfigured security group or wrong IAM policy can cause outages or security breaches that take hours to diagnose.

The Solution: Pulumi lets you write tests in the same language as your infrastructure code, catching errors before they reach production.

Real Impact: Organizations with tested IaC report 90% fewer infrastructure-related incidents and 5x faster rollback times.

Real-World Analogy

Think of testing Pulumi code like testing a building before people move in:

Unit Tests = Inspecting individual components (wiring, plumbing) before installation
Property Tests = Checking that every room meets building codes (no public S3 buckets)
Integration Tests = Walking through the entire building to verify everything works together
Mocks = Using architectural models instead of building real structures for each test

The Testing Pyramid for IaC

Unit Tests

Test individual resource configurations using mocks. Fast, cheap, and catch configuration errors early.

Property Tests

Validate resource properties across your entire stack. Ensure compliance policies are met before deployment.

Integration Tests

Deploy real infrastructure in a test environment using the Automation API and verify it works end-to-end.

Policy Tests

Use CrossGuard policy packs to enforce organizational standards across all Pulumi programs.

Testing Pyramid for Infrastructure as Code

Key Takeaway: Pulumi supports three testing levels: unit tests (fast, mock cloud APIs, test program logic), property tests (validate resource configurations during preview), and integration tests (deploy real resources, run assertions, then destroy). Use all three for comprehensive infrastructure validation.

Unit Testing

Setting Up Mocks

index.test.ts

import * as pulumi from "@pulumi/pulumi";
import { describe, it, expect, beforeAll } from "vitest";

// Mock Pulumi runtime before importing resources
pulumi.runtime.setMocks({
    newResource: (args: pulumi.runtime.MockResourceArgs) => {
        return {
            id: `${args.name}-id`,
            state: {
                ...args.inputs,
                arn: `arn:aws:s3:::${args.inputs.bucket || args.name}`,
            },
        };
    },
    call: (args: pulumi.runtime.MockCallArgs) => {
        return args.inputs;
    },
});

describe("S3 Bucket", () => {
    let infra: typeof import("./index");

    beforeAll(async () => {
        infra = await import("./index");
    });

    it("should have versioning enabled", async () => {
        const versioning = await new Promise(resolve =>
            infra.bucketVersioning.apply(v => resolve(v))
        );
        expect(versioning).toBe("Enabled");
    });

    it("should block public access", async () => {
        const blockPublic = await new Promise(resolve =>
            infra.publicAccessBlocked.apply(v => resolve(v))
        );
        expect(blockPublic).toBe(true);
    });
});

Output (unit test)

$ python -m pytest tests/test_infra.py -v
tests/test_infra.py::test_s3_bucket_is_private PASSED
tests/test_infra.py::test_rds_is_encrypted PASSED
tests/test_infra.py::test_security_group_no_public_ssh PASSED
tests/test_infra.py::test_instance_type_is_valid PASSED

4 passed in 0.8s (no cloud resources created)

Common Mistake

Wrong: Writing integration tests that create resources but do not clean up on failure

Why it fails: If an assertion fails mid-test, the test exits without destroying resources. Orphaned cloud resources accumulate, costing money and creating security risks.

Instead: Use try/finally or Pulumi's test framework which automatically runs pulumi destroy regardless of test outcome. Set resource tags with a TTL and run a cleanup job for any resources older than their TTL.

Property Testing

Validating Resource Properties

property-tests.ts

import * as policy from "@pulumi/policy";

const stackPolicy = new policy.PolicyPack("tests", {
    policies: [
        {
            name: "s3-no-public-read",
            description: "S3 buckets must not allow public read",
            enforcementLevel: "mandatory",
            validateResource: policy.validateResourceOfType(
                aws.s3.BucketV2,
                (bucket, args, reportViolation) => {
                    // Check property tests on each S3 bucket
                }
            ),
        },
        {
            name: "ec2-must-have-tags",
            description: "All EC2 instances must have required tags",
            enforcementLevel: "mandatory",
            validateResource: policy.validateResourceOfType(
                aws.ec2.Instance,
                (instance, args, reportViolation) => {
                    if (!instance.tags || !instance.tags["Environment"]) {
                        reportViolation("EC2 instances must have an 'Environment' tag");
                    }
                    if (!instance.tags || !instance.tags["Team"]) {
                        reportViolation("EC2 instances must have a 'Team' tag");
                    }
                }
            ),
        },
    ],
});

Integration Testing

Using the Automation API

integration.test.ts

import { LocalWorkspace } from "@pulumi/pulumi/automation";
import { describe, it, expect, afterAll } from "vitest";

describe("Infrastructure Integration", () => {
    let stack: any;

    it("should deploy successfully", async () => {
        stack = await LocalWorkspace.createOrSelectStack({
            stackName: "test",
            workDir: "./infra",
        });

        await stack.setConfig("aws:region", { value: "us-east-1" });

        const upResult = await stack.up({ onOutput: console.log });
        expect(upResult.summary.result).toBe("succeeded");
    }, 300000);

    it("should have correct outputs", async () => {
        const outputs = await stack.outputs();
        expect(outputs.bucketName.value).toBeDefined();
        expect(outputs.vpcId.value).toMatch(/^vpc-/);
    });

    afterAll(async () => {
        if (stack) {
            await stack.destroy({ onOutput: console.log });
            await stack.workspace.removeStack("test");
        }
    });
});

Mocking Resources

advanced-mocks.ts

// Advanced mock that simulates realistic resource behavior
pulumi.runtime.setMocks({
    newResource: (args) => {
        switch (args.type) {
            case "aws:ec2/instance:Instance":
                return {
                    id: "i-1234567890abcdef0",
                    state: {
                        ...args.inputs,
                        publicIp: "54.123.45.67",
                        privateIp: "10.0.1.100",
                        arn: `arn:aws:ec2:us-east-1:123456789:instance/${args.name}`,
                    },
                };
            case "aws:s3/bucketV2:BucketV2":
                return {
                    id: args.name,
                    state: {
                        ...args.inputs,
                        bucket: `${args.name}-abc123`,
                        arn: `arn:aws:s3:::${args.name}`,
                    },
                };
            default:
                return { id: `${args.name}-id`, state: args.inputs };
        }
    },
    call: (args) => args.inputs,
});

Deep Dive: Mocking Pulumi Resources

Pulumi's mocking framework intercepts resource creation during tests and returns fake values instead of calling cloud APIs. This makes unit tests fast (milliseconds) and free (no cloud costs). Set pulumi.runtime.set_mocks(MyMocks()) at the start of your test, where MyMocks implements new_resource() to return fake IDs and properties, and call() to mock provider function calls. Test that your program creates the right resources with the right properties -- leave testing of actual cloud behavior to integration tests.

Quick Reference

Test Type	Speed	Real Resources?	Best For
Unit Tests	Milliseconds	No (mocked)	Configuration validation, logic checks
Property Tests	Seconds	No (policy)	Compliance, tagging, security rules
Integration Tests	Minutes	Yes (deployed)	End-to-end verification, smoke tests
Policy (CrossGuard)	Seconds	No (pre-deploy)	Organization-wide enforcement

Testing Best Practices

Run unit tests on every commit - they are fast and catch most issues
Use property tests to enforce company-wide compliance policies
Run integration tests in CI/CD against an isolated test account
Always clean up test stacks to avoid resource leaks and costs
Use pulumi.runtime.setMocks() to avoid real API calls in unit tests