CICD on Bastab C

Infrastructure as Code with AWS CDK - Series Overview

Mon, 21 Apr 2025 00:00:00 +0000

AWS CDK lets you define cloud infrastructure in familiar programming languages - Python, TypeScript, Java - and synthesise it into CloudFormation. Compared to writing raw CloudFormation or Terraform HCL, CDK gives you loops, conditionals, type safety, and reusable constructs.

This series covers practical CDK patterns using Python, with a consistent use case across all parts so the trade-offs are easy to compare.

Why CDK over CloudFormation for AWS-native workloads?

CloudFormation is YAML or JSON - no loops, no conditionals, no abstraction. Any shared pattern gets copy-pasted.
CDK uses a real programming language - loops, functions, classes, and type checks all apply to infrastructure the same way they apply to application code.
L2 constructs handle boilerplate. bucket.grant_read(fn) generates the IAM policy, role attachment, and resource reference in one call. The CloudFormation equivalent is four resources wired together manually.
The compiler catches mistakes before CloudFormation ever sees the template.
CDK still synthesises to CloudFormation under the hood - rollbacks, drift detection, and stack history are unchanged.

Why CDK over Terraform for AWS-native workloads?

Terraform’s strength is multi-cloud. For AWS-only workloads, that comes with overhead that doesn’t pay off - a state backend, a provider version to pin, and HCL alongside the application code.
CDK uses CloudFormation as the deployment engine - AWS manages state natively. No S3 bucket for state, no DynamoDB table for locking, no terraform init in the pipeline.
New AWS services appear in CDK constructs faster.
IAM is easier. Terraform requires writing policy JSON by hand and threading ARNs between resources. CDK’s grant_* methods generate least-privilege policies from the resource graph.
Terraform is the right call when infrastructure spans multiple cloud providers, or when the team already has a mature Terraform codebase.

Use case

Deploy an AWS Lambda function that processes files uploaded to an S3 bucket. Configuration - bucket name, log level, Lambda timeout - varies per environment (dev, staging, prod).

Simple infrastructure by design - the CDK patterns are the point, not the resources.

The series

Chapter 1 - Project setup and bootstrapping

Getting a CDK project off the ground - directory structure, virtual environments, bootstrapping an AWS account, and understanding the synth/deploy cycle.

Chapter 2 - Managing configuration and context

How environment-specific values flow into CDK stacks - four approaches covering local static config, local dynamic config, SSM Parameter Store, and Secrets Manager, with trade-offs and code examples for each.

Building L3 constructs - extracting resources into reusable units, the keyword-only props pattern, and sharing constructs across stacks.

Chapter 4 - Testing CDK stacks

Unit testing constructs and stacks with aws_cdk.assertions - fine-grained assertions, snapshot testing, and when each applies.

Chapter 5 - CI/CD with CDK

Automating CDK deployments with GitHub Actions, Jenkins, and Azure DevOps - auth setup, multi-environment promotion, and approval gates.

Notes

All examples use Python and CDK v2.
Steps are tested on WSL2/Ubuntu on Windows and native macOS terminal.
More chapters added as I work through these patterns.

AWS Kiro CLI - Manage AWS Infrastructure from the Terminal

Sat, 02 May 2026 00:00:00 +0000

Kiro CLI is an AI-assisted terminal tool for AWS, rebranded from Amazon Q Developer CLI in November 2025. It sits on top of your existing AWS credentials and tooling. The q and q chat shortcuts from Q CLI still work but kiro-cli is the current entry point.

Install

macOS

brew install --cask kiro-cli

Or via script:

curl -fsSL https://cli.kiro.dev/install | bash

Windows (PowerShell)

irm 'https://cli.kiro.dev/install.ps1' | iex

Linux / WSL2 (Ubuntu / Debian) - .deb

wget https://desktop-release.q.us-east-1.amazonaws.com/latest/kiro-cli.deb
sudo dpkg -i kiro-cli.deb
sudo apt-get install -f

Linux - ZIP (x86-64, non-Debian distros)

curl --proto '=https' --tlsv1.2 -sSf \
  'https://desktop-release.q.us-east-1.amazonaws.com/latest/kirocli-x86_64-linux.zip' \
  -o kirocli.zip
unzip kirocli.zip
./kirocli/install.sh

Verify:

kiro-cli version
kiro-cli doctor   # diagnoses config and auth issues

Authenticate

kiro-cli login     # opens browser to complete sign-in
kiro-cli whoami    # shows current user and auth method
kiro-cli logout

AWS Builder ID - free individual account at builder.aws. No existing AWS account required. Enough to get started with 50 free credits/month (500 bonus credits on first sign-in).

Builder ID with AWS accounts - You can log in to Kiro with a personal Builder ID for the AI features, and connect to your AWS account using AWS SSO profile. Kiro picks up AWS credentials are active in the environment:

Set the profile for the session and run commands targeting the profile:

export AWS_PROFILE=my-prod
aws sts get-caller-identity              # confirm which account is active
aws sts get-caller-identity --profile my-prod
kiro-cli chat "get Compute Optimizer recommendations"

Key commands

kiro-cli with no arguments opens an interactive session - type prompts directly without any prefix. kiro-cli chat "prompt" is for scripted or one-shot use.

kiro-cli                          # interactive session - type prompts directly
kiro-cli chat "your prompt"       # start session with an initial prompt
kiro-cli chat --resume            # continue previous session
kiro-cli chat --no-interactive "prompt"  # one-shot, no interaction - useful in scripts
kiro-cli translate "list S3 buckets"    # natural language to shell command
kiro-cli agent list               # list available agents
kiro-cli mcp add                  # add an MCP server
kiro-cli mcp list                 # list configured MCP servers
kiro-cli mcp status               # check MCP server health
kiro-cli settings list            # view current settings
kiro-cli update                   # upgrade to latest version

Infrastructure workflows

Kiro CLI does not call AWS APIs directly - it generates and optionally runs commands using your existing AWS credentials. A few patterns that work well:

Generate and run AWS CLI commands from natural language:

kiro-cli translate "list all EC2 instances in us-east-1 with their state"
# outputs: aws ec2 describe-instances --region us-east-1 --query ...

Pipe AWS CLI output into chat for analysis:

aws cloudwatch describe-alarms --state-value ALARM | \
  kiro-cli chat --no-interactive "summarise these alarms and suggest remediation"

Generate Terraform or CloudFormation:

kiro-cli chat "write a Terraform module for a private S3 bucket with versioning and a 90-day lifecycle rule to Glacier"

Use in CI without interaction:

KIRO_API_KEY=ksk_xxxxxxxx kiro-cli chat --no-interactive \
  "explain this deployment error: $(cat deploy.log | tail -50)"

Run commands with auto-approval in trusted environments:

kiro-cli chat --trust-all-tools "get Compute Optimizer recommendations for my account"

Notes

If migrating from Q CLI, config migrates automatically from ~/.aws/amazonq to ~/.kiro on first run.
kiro-cli doctor is the first thing to run if auth or config behaves unexpectedly.
The CLI is not open source - the license changed from Apache to AWS Intellectual Property License with the Kiro rebrand.

GitHub PR Fix: "Commits must have verified signatures" blocking PR merge

Sat, 25 Apr 2026 00:00:00 +0000

Problem

GitHub branch protection requires all commits to have verified GPG signatures. Two commits at the base of the branch (made before GPG signing was configured) were unsigned, blocking the merge.

Root cause

Commits made before commit.gpgsign=true was configured in git have no GPG signature. GitHub requires all commits in a PR to be verified - one unsigned commit blocks the merge regardless of approvals.

Steps to fix

1. Identify unsigned commits

git log --format="%G? %ad %s" --date=short origin/main..HEAD

Look for lines starting with N (no signature). Note the hash of the oldest unsigned commit’s parent on main.

2. Re-sign all branch commits using filter-branch

git filter-branch -f --commit-filter 'git commit-tree -S "$@"' -- origin/main..HEAD

Rewrites all branch commits adding a GPG signature - no content changes, no conflicts.

3. Verify no unsigned commits remain

git log --format="%G?" origin/main..HEAD | Select-String "^N$"

Should return nothing.

4. Force push

Attempt force push via terminal using the configured remote:

git push --force origin <branch>

If this fails with “repository not found” (HTTPS remote, no credentials) or “Could not read from remote repository” (SSH not configured), use a classic PAT with repo scope directly in the URL:

git push --force https://<username>:<PAT>@github.com/<org>/<repo>.git <branch>

5. Update local tracking ref (if GitHub Desktop shows stale state)

git update-ref refs/remotes/origin/<branch> <tip-commit-hash>

Notes

Avoid git rebase --exec "git commit --amend -S" if the branch has merge commits - it drops them and causes conflicts.
Avoid git rebase --rebase-merges - re-applying old merge commits causes conflicts from the original merges.
filter-branch rewrites commit objects without replaying merges, which is what’s needed here.
Fine-grained PATs do not work for org repositories - use classic PAT only.
After force pushing, avoid git pull / IDE auto-sync until the PR is merged - it will re-introduce the old unsigned commits via a new merge.

AWS CDK - CI/CD Pipelines

Sun, 27 Jul 2025 00:00:00 +0000

This is Chapter 5 of the Infrastructure as Code with AWS CDK series. The same cdk deploy command that runs locally works in any pipeline - the differences are auth and orchestration.

Common flags for CI

These apply regardless of which CI system you use:

cdk synth                          # generate CloudFormation template
cdk deploy --require-approval never -c env=prod   # deploy without interactive prompts
cdk deploy --outputs-file outputs.json            # write stack outputs to file

--require-approval never skips the IAM change confirmation that CDK shows interactively. Required in CI.

-c env=prod passes the environment context from Chapter 2.

GitHub Actions

Auth

OIDC is the recommended auth method - no long-lived credentials stored in GitHub. The pipeline assumes an IAM role directly via GitHub’s OIDC provider.

Create the OIDC provider in AWS once per account:

aws iam create-open-id-connect-provider \
  --url https://token.actions.githubusercontent.com \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list 6938fd4d98bab03faadb97b34396831e3780aea1

Create an IAM role with a trust policy scoped to your repository:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringLike": {
        "token.actions.githubusercontent.com:sub": "repo:my-org/my-repo:*"
      }
    }
  }]
}

Attach whatever IAM policies the CDK deploy needs (CloudFormation, S3, Lambda, IAM) to this role.

Workflow

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

jobs:
  deploy-dev:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install CDK CLI
        run: npm install -g aws-cdk

      - name: Install dependencies
        run: pip install -r requirements.txt -r requirements-dev.txt

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
          aws-region: us-east-1

      - name: Synth
        run: cdk synth -c env=dev

      - name: Test
        run: pytest

      - name: Deploy dev
        run: cdk deploy --require-approval never -c env=dev

  deploy-prod:
    runs-on: ubuntu-latest
    needs: deploy-dev
    environment: production
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install CDK CLI
        run: npm install -g aws-cdk

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsRole
          aws-region: us-east-1

      - name: Deploy prod
        run: cdk deploy --require-approval never -c env=prod

The environment: production job pauses for a manual approval before running. Configure required reviewers under Settings > Environments > production in the GitHub repository.

Jenkins

Auth

The simplest setup: run the Jenkins agent on an EC2 instance with an IAM instance role. The CDK CLI picks up credentials automatically via the instance metadata service - no credentials to manage.

If the agent is not on EC2, store AWS credentials as a Jenkins credential (Credentials > Add > AWS Credentials) and inject them into the pipeline with the withCredentials block.

Jenkinsfile

pipeline {
    agent any

    environment {
        AWS_DEFAULT_REGION = 'us-east-1'
    }

    stages {
        stage('Install') {
            steps {
                sh 'pip install -r requirements.txt -r requirements-dev.txt'
                sh 'npm install -g aws-cdk'
            }
        }

        stage('Synth') {
            steps {
                sh 'cdk synth -c env=dev'
            }
        }

        stage('Test') {
            steps {
                sh 'pytest'
            }
        }

        stage('Deploy dev') {
            steps {
                sh 'cdk deploy --require-approval never -c env=dev'
            }
        }

        stage('Approve prod') {
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
            }
        }

        stage('Deploy prod') {
            steps {
                sh 'cdk deploy --require-approval never -c env=prod'
            }
        }
    }

    post {
        failure {
            echo 'Pipeline failed - check CloudFormation events for rollback details'
        }
    }
}

The input step pauses the pipeline until someone clicks Deploy in the Jenkins UI.

If credentials are stored in Jenkins rather than on an instance role, wrap the deploy steps:

withCredentials([[
    $class: 'AmazonWebServicesCredentialsBinding',
    credentialsId: 'aws-deploy-credentials'
]]) {
    sh 'cdk deploy --require-approval never -c env=prod'
}

Azure DevOps

Auth

Store AWS credentials as secret pipeline variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) or use the AWS Toolkit for Azure DevOps extension, which adds an AWSShellScript task with a configured service connection.

The example below uses secret variables directly, which works without the extension.

Pipeline

# azure-pipelines.yml
trigger:
  branches:
    include:
      - main

variables:
  pythonVersion: "3.12"
  awsRegion: "us-east-1"

stages:
  - stage: DeployDev
    displayName: Deploy to dev
    jobs:
      - job: Deploy
        pool:
          vmImage: ubuntu-latest
        steps:
          - task: UsePythonVersion@0
            inputs:
              versionSpec: $(pythonVersion)

          - script: |
              pip install -r requirements.txt -r requirements-dev.txt
              npm install -g aws-cdk
            displayName: Install dependencies

          - script: cdk synth -c env=dev
            displayName: Synth
            env:
              AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
              AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)
              AWS_DEFAULT_REGION: $(awsRegion)

          - script: pytest
            displayName: Test

          - script: cdk deploy --require-approval never -c env=dev
            displayName: Deploy dev
            env:
              AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
              AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)
              AWS_DEFAULT_REGION: $(awsRegion)

  - stage: DeployProd
    displayName: Deploy to prod
    dependsOn: DeployDev
    jobs:
      - deployment: Deploy
        environment: production
        pool:
          vmImage: ubuntu-latest
        strategy:
          runOnce:
            deploy:
              steps:
                - checkout: self

                - task: UsePythonVersion@0
                  inputs:
                    versionSpec: $(pythonVersion)

                - script: |
                    pip install -r requirements.txt
                    npm install -g aws-cdk
                  displayName: Install dependencies

                - script: cdk deploy --require-approval never -c env=prod
                  displayName: Deploy prod
                  env:
                    AWS_ACCESS_KEY_ID: $(AWS_ACCESS_KEY_ID)
                    AWS_SECRET_ACCESS_KEY: $(AWS_SECRET_ACCESS_KEY)
                    AWS_DEFAULT_REGION: $(awsRegion)

The deployment job type with environment: production enables approval gates. Configure approvers under Pipelines > Environments > production in Azure DevOps.

AWS CodePipeline

CDK ships a pipelines module that creates a CodePipeline pipeline directly from CDK code:

from aws_cdk import pipelines

pipeline = pipelines.CodePipeline(self, "Pipeline",
    synth=pipelines.ShellStep("Synth",
        input=pipelines.CodePipelineSource.git_hub("my-org/my-repo", "main"),
        commands=[
            "pip install -r requirements.txt",
            "npm install -g aws-cdk",
            "cdk synth"
        ]
    )
)

The pipeline is self-mutating - when CDK code changes, the pipeline updates itself before deploying application changes. It is tightly coupled to CodePipeline and requires more upfront setup than the external CI approaches above, but is the right fit when the deployment needs to stay fully within AWS.

Notes

--require-approval never skips the IAM diff prompt. Review IAM changes in cdk diff output or pull request checks before merging rather than at deploy time.
The cdk.context.json file should be committed - pipelines need it for SSM lookups cached in Chapter 2 to avoid live SSM calls on every run.
CloudFormation automatically rolls back a failed deployment to the previous stable state. No extra rollback configuration is needed in the pipeline.
For OIDC in GitHub Actions, scope the trust condition to a specific branch (repo:my-org/my-repo:ref:refs/heads/main) rather than * to limit which workflows can assume the role.

AWS CDK - Testing CDK Stacks

Sat, 05 Jul 2025 00:00:00 +0000

This is Chapter 4 of the Infrastructure as Code with AWS CDK series. Chapter 3 built FileProcessorConstruct - the tests here cover that construct and the stack it lives in.

How CDK testing works

CDK unit tests don’t deploy anything. They synthesize a stack into a CloudFormation template and run assertions against that template. No AWS credentials needed, and the suite runs in CI without any special setup.

The aws_cdk.assertions module is already part of aws-cdk-lib - no extra install required.

Setup

cdk init adds pytest to requirements-dev.txt automatically. Install it if not already active:

pip install -r requirements-dev.txt

Create the test files:

my-cdk-app/
├── tests/
│   ├── __init__.py
│   ├── test_file_processor.py
│   └── test_stack.py

Testing the construct

Each test builds a minimal stack, instantiates the construct inside it, and asserts against the synthesized template:

# tests/test_file_processor.py
import pytest
from aws_cdk import App, Stack
from aws_cdk.assertions import Template, Match
from my_cdk_app.file_processor import FileProcessorConstruct

TEST_PROPS = dict(
    bucket_name="test-bucket",
    log_level="DEBUG",
    lambda_timeout=30,
)


@pytest.fixture
def template():
    app = App()
    stack = Stack(app, "TestStack")
    FileProcessorConstruct(stack, "TestConstruct", **TEST_PROPS)
    return Template.from_stack(stack)


def test_bucket_created(template):
    template.has_resource_properties("AWS::S3::Bucket", {
        "BucketName": "test-bucket"
    })


def test_lambda_runtime_and_timeout(template):
    template.has_resource_properties("AWS::Lambda::Function", {
        "Runtime": "python3.12",
        "Timeout": 30,
        "Environment": {
            "Variables": {"LOG_LEVEL": "DEBUG"}
        }
    })


def test_lambda_has_read_access(template):
    template.has_resource_properties("AWS::IAM::Policy", {
        "PolicyDocument": {
            "Statement": Match.array_with([
                Match.object_like({
                    "Action": Match.array_with(["s3:GetObject*"]),
                    "Effect": "Allow"
                })
            ])
        }
    })

has_resource_properties does a subset match - the resource only needs to contain the specified properties, not match them exactly. Match.array_with and Match.object_like apply the same subset logic to nested arrays and objects.

Testing the stack

Test MyStack the same way, using a config dict that matches what app.py passes:

# tests/test_stack.py
from aws_cdk import App
from aws_cdk.assertions import Template
from my_cdk_app.my_cdk_app_stack import MyStack

TEST_CONFIG = {
    "bucket_name": "test-bucket",
    "log_level": "INFO",
    "lambda_timeout": 60,
}


def test_stack_resource_count():
    app = App()
    stack = MyStack(app, "TestStack", config=TEST_CONFIG)
    template = Template.from_stack(stack)
    template.resource_count_is("AWS::S3::Bucket", 1)
    template.resource_count_is("AWS::Lambda::Function", 1)

Snapshot testing

Snapshot tests capture the full synthesized template and fail if anything changes. Write the template to a file on first run, then compare on subsequent runs:

import json
from pathlib import Path
from aws_cdk import App
from aws_cdk.assertions import Template
from my_cdk_app.my_cdk_app_stack import MyStack

SNAPSHOT_PATH = Path(__file__).parent / "snapshots" / "stack.json"

TEST_CONFIG = {
    "bucket_name": "test-bucket",
    "log_level": "INFO",
    "lambda_timeout": 60,
}


def test_stack_snapshot():
    app = App()
    stack = MyStack(app, "TestStack", config=TEST_CONFIG)
    template = Template.from_stack(stack).to_json()

    if not SNAPSHOT_PATH.exists():
        SNAPSHOT_PATH.parent.mkdir(exist_ok=True)
        SNAPSHOT_PATH.write_text(json.dumps(template, indent=2))
        return

    expected = json.loads(SNAPSHOT_PATH.read_text())
    assert template == expected

Commit the snapshot file. To update it after an intentional change, delete the file and run the test once to regenerate.

The downside: CDK version bumps routinely touch metadata in the template - aws:cdk:path, asset hashes, checksums. Any of these invalidates the snapshot even when nothing meaningful changed. They work well for catching unintended IAM or resource config changes, but generate noise as a general regression check.

Running tests

pytest
pytest -v                             # verbose output
pytest tests/test_file_processor.py  # single file

Notes

has_resource_properties only checks the Properties block. To assert on the full resource including DeletionPolicy, DependsOn, or Metadata, use has_resource instead.
Template.from_stack calls app.synth() implicitly - no need to call it manually.
CDK assigns logical IDs based on the construct path. Changing a construct’s id argument changes its logical ID, which CloudFormation treats as a delete and recreate. Snapshot tests catch this; has_resource_properties tests do not.
For TypeScript CDK projects, Jest has built-in snapshot support via toMatchSnapshot(). Python has no equivalent - the manual file approach above is the workaround.

AWS CDK - Writing and Sharing Constructs

Sat, 14 Jun 2025 00:00:00 +0000

This is Chapter 3 of the Infrastructure as Code with AWS CDK series. Chapters 1 and 2 put the S3 bucket and Lambda directly in MyStack. Both get extracted into a reusable construct here.

L1, L2, L3

CDK organises constructs into three levels:

L1 - raw CloudFormation resource wrappers, prefixed with Cfn. CfnBucket, CfnFunction. No defaults added - every property must be set explicitly.
L2 - CDK’s built-in higher-level constructs. s3.Bucket, _lambda.Function. These set sensible defaults, expose helper methods, and handle IAM via grant_* methods.
L3 - composite constructs you write that bundle multiple resources into a single unit. The previous chapters used L2 constructs directly inside a stack. This chapter builds an L3.

The problem

The current MyStack creates the bucket and Lambda directly:

class MyStack(Stack):
    def __init__(self, scope, id, config, **kwargs):
        super().__init__(scope, id, **kwargs)

        bucket = s3.Bucket(self, "FilesBucket",
            bucket_name=config["bucket_name"]
        )

        fn = _lambda.Function(self, "ProcessorFunction",
            runtime=_lambda.Runtime.PYTHON_3_12,
            handler="handler.handler",
            code=_lambda.Code.from_asset("lambda"),
            timeout=Duration.seconds(config["lambda_timeout"]),
            environment={"LOG_LEVEL": config["log_level"]}
        )

        bucket.grant_read(fn)

If you need a second processor - say one for raw uploads and one for processed files - all of this gets duplicated. That’s what a construct solves.

Writing the construct

Add file_processor.py inside the existing app module:

my-cdk-app/
├── app.py
├── my_cdk_app/
│   ├── __init__.py
│   ├── my_cdk_app_stack.py
│   └── file_processor.py     ← new

# my_cdk_app/file_processor.py
from aws_cdk import (
    Duration,
    aws_s3 as s3,
    aws_lambda as _lambda,
)
from constructs import Construct


class FileProcessorConstruct(Construct):
    def __init__(
        self,
        scope: Construct,
        id: str,
        *,
        bucket_name: str,
        log_level: str,
        lambda_timeout: int,
    ):
        super().__init__(scope, id)

        self.bucket = s3.Bucket(self, "Bucket",
            bucket_name=bucket_name
        )

        self.fn = _lambda.Function(self, "Function",
            runtime=_lambda.Runtime.PYTHON_3_12,
            handler="handler.handler",
            code=_lambda.Code.from_asset("lambda"),
            timeout=Duration.seconds(lambda_timeout),
            environment={"LOG_LEVEL": log_level}
        )

        self.bucket.grant_read(self.fn)

The * before the keyword arguments makes them keyword-only - callers must pass them by name, not by position. self.bucket and self.fn are public so callers can reach in and extend if needed.

Using it in a stack

# my_cdk_app/my_cdk_app_stack.py
from aws_cdk import Stack
from constructs import Construct
from my_cdk_app.file_processor import FileProcessorConstruct


class MyStack(Stack):
    def __init__(self, scope: Construct, id: str, config: dict, **kwargs):
        super().__init__(scope, id, **kwargs)

        FileProcessorConstruct(self, "FileProcessor",
            bucket_name=config["bucket_name"],
            log_level=config["log_level"],
            lambda_timeout=config["lambda_timeout"]
        )

Reusing the construct

Two processors in the same stack, no duplication:

class MyStack(Stack):
    def __init__(self, scope: Construct, id: str, config: dict, **kwargs):
        super().__init__(scope, id, **kwargs)

        FileProcessorConstruct(self, "RawProcessor",
            bucket_name=f"{config['bucket_name']}-raw",
            log_level=config["log_level"],
            lambda_timeout=config["lambda_timeout"]
        )

        FileProcessorConstruct(self, "ProcessedProcessor",
            bucket_name=f"{config['bucket_name']}-processed",
            log_level=config["log_level"],
            lambda_timeout=config["lambda_timeout"]
        )

CDK uses the id argument (RawProcessor, ProcessedProcessor) to make all resource logical IDs unique within the stack. Both constructs deploy without naming conflicts.

Using exposed attributes

Use self.bucket and self.fn when something needs to be added at the call site:

processor = FileProcessorConstruct(self, "FileProcessor",
    bucket_name=config["bucket_name"],
    log_level=config["log_level"],
    lambda_timeout=config["lambda_timeout"]
)

# Grant a second function read access to the same bucket
processor.bucket.grant_read(reporting_fn)

If it’s specific to one stack, keep it there rather than baking it into the construct.

Notes

Do not name the constructs directory constructs/ - it conflicts with the CDK constructs package import. Putting files inside the app module (my_cdk_app/) sidesteps this entirely.
The * in the __init__ signature enforces keyword arguments. Without it, callers can pass values positionally, which gets confusing with three or more parameters.
self.bucket and self.fn being public is a choice. If the construct is fully self-contained and callers should never reach in, keep them private with a leading underscore (self._bucket).
To use a construct across multiple CDK projects, move it into a standalone Python package and install it as a dependency. Cross-language support (TypeScript, Java) requires jsii - a separate topic.

AWS CDK - Managing Configuration and Context

Wed, 28 May 2025 00:00:00 +0000

This is Chapter 2 of the Infrastructure as Code with AWS CDK series. Four approaches to environment-specific configuration in CDK stacks - same use case throughout so the trade-offs sit side by side.

Use case: An S3 bucket and Lambda function where bucket name, log level, and Lambda timeout vary per environment (dev, staging, prod).

Quick comparison

Approach	Version controlled	Supports secrets	Change without redeploy	Shared across stacks
Static config (`cdk.json`)	Yes	No	No	No
Dynamic config	No	Partial	Yes	No
Secrets Manager	No	Yes	Yes	Yes
CI/CD context injection	No	No	Yes	No

Local dev patterns

Suitable for personal projects or local development. Neither is a good fit for CI/CD pipelines or shared team environments.

Static config (`cdk.json`)

Store environment-specific config in cdk.json under the context key. CDK reads this file automatically at synth time.

When to use: Stable, non-sensitive values that are the same for everyone on the team.

{
  "app": "python3 app.py",
  "context": {
    "dev": {
      "bucket_name": "my-bucket-dev",
      "log_level": "DEBUG",
      "lambda_timeout": 30
    },
    "prod": {
      "bucket_name": "my-bucket-prod",
      "log_level": "INFO",
      "lambda_timeout": 60
    }
  }
}

Read context in app.py:

app = cdk.App()
env_name = app.node.try_get_context("env") or "dev"
config = app.node.try_get_context(env_name)

MyStack(app, f"MyStack-{env_name}", config=config)

Deploy with:

cdk deploy -c env=dev
cdk deploy -c env=prod

Trade-off: Any config change requires a commit and redeploy. Never put secrets here - cdk.json is committed to source control.

Dynamic config

Load config at synth time from a local file or environment variables, outside the CDK project. Useful when values differ per developer or machine.

When to use: Values that differ between developers - e.g. personal AWS account IDs, local endpoint overrides.

Create a config.dev.json outside source control:

{
  "bucket_name": "my-bucket-dev",
  "log_level": "DEBUG",
  "lambda_timeout": 30
}

Load it in app.py:

import json
import os

app = cdk.App()
env_name = os.environ.get("ENV", "dev")

with open(f"config.{env_name}.json") as f:
    config = json.load(f)

MyStack(app, f"MyStack-{env_name}", config=config)

Add config files to .gitignore:

config.*.json

Deploy with:

ENV=dev cdk deploy
ENV=prod cdk deploy

Trade-off: Config lives outside source control - harder to audit and reproduce. Not suitable for team environments where config needs to be consistent.

Production patterns

Config lives in AWS or is injected by the pipeline - not in the codebase. Values can change without a code commit.

Secrets Manager

Store sensitive config in Secrets Manager. Values are fetched at deploy time and injected into the Lambda environment - they are never visible in the CloudFormation template.

When to use: API keys, database passwords, tokens. Secrets should never go in cdk.json or plain environment variables.

Create a secret (run once per environment):

aws secretsmanager create-secret \
  --name "/my-app/dev/config" \
  --secret-string '{"api_key":"abc123","db_password":"s3cr3t"}'

Reference the secret in the stack:

from aws_cdk import aws_secretsmanager as secretsmanager

class MyStack(Stack):
    def __init__(self, scope, id, env_name, **kwargs):
        super().__init__(scope, id, **kwargs)

        secret = secretsmanager.Secret.from_secret_name_v2(
            self, "AppSecret", f"/my-app/{env_name}/config"
        )

        fn = _lambda.Function(self, "MyFunction",
            ...
            environment={
                "SECRET_ARN": secret.secret_arn
            }
        )

        secret.grant_read(fn)

The Lambda reads the secret at runtime using the ARN:

import boto3, json, os

def handler(event, context):
    client = boto3.client("secretsmanager")
    secret = json.loads(client.get_secret_value(
        SecretId=os.environ["SECRET_ARN"]
    )["SecretString"])
    api_key = secret["api_key"]

Trade-off: Cost per secret per month. Values are only available at runtime, not synth time. Rotation adds operational overhead but is worth setting up for credentials.

For non-sensitive shared config that needs to change without a code commit - bucket names, log levels, timeouts - SSM Parameter Store is an alternative. Parameters are fetched at synth time via value_from_lookup and cached in cdk.context.json:

from aws_cdk import aws_ssm as ssm

bucket_name = ssm.StringParameter.value_from_lookup(
    self, f"/my-app/{env_name}/bucket_name"
)

Commit cdk.context.json so CI doesn’t need live SSM access on every run. SSM has no secrets support - keep anything sensitive in Secrets Manager.

CI/CD context injection

Pass values into the stack directly from the pipeline via CDK context flags. The stack reads them at synth time - no external config files or AWS lookups required.

When to use: Values that are known to the pipeline at deploy time - environment name, account ID, deploy target region, feature flags. Works well when the pipeline is the source of truth for what gets deployed where.

The stack reads context with try_get_context and uses Stack.of(self).account for the resolved AWS account:

from aws_cdk import Stack
from constructs import Construct

class MyStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        env_name = self.node.try_get_context("env")
        profile = self.node.try_get_context("profile")
        account_id = Stack.of(self).account

        bucket = s3.Bucket(self, "FilesBucket",
            bucket_name=f"my-bucket-{env_name}-{account_id}"
        )

The pipeline passes context at deploy time:

# GitHub Actions, CodePipeline, or any CI runner
cdk deploy -c env=prod -c profile=my-prod

A GitHub Actions step looks like:

- name: Deploy
  run: cdk deploy --require-approval never -c env=prod -c profile=my-prod
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

try_get_context returns None if the key is not passed. Add a guard if the value is required:

env_name = self.node.try_get_context("env")
if not env_name:
    raise ValueError("env context is required - pass with -c env=<name>")

Trade-off: Context values are visible in the pipeline logs and the synthesized CloudFormation template. Keep secrets in Secrets Manager and pass only non-sensitive identifiers this way.

Notes

These approaches combine well - SSM for non-sensitive shared config, Secrets Manager for secrets, CI/CD context injection for environment identifiers.
Secrets Manager values are never embedded in CloudFormation templates - they are resolved at runtime by the Lambda itself.
On Windows with PowerShell, set the env var with $env:ENV="dev"; cdk deploy instead of the bash syntax.

AWS CDK - Project Setup and Bootstrapping

Sun, 11 May 2025 00:00:00 +0000

This is Chapter 1 of the Infrastructure as Code with AWS CDK series. Two things come before writing any stack code: initialising the CDK project and bootstrapping the AWS account.

Prerequisites

Python 3.9+
Node.js 18+ (CDK CLI is a Node package)
AWS CLI v2 configured with valid credentials - see Connect to AWS SSO and SSH into EC2 Instance
An AWS account with permissions to create IAM roles, S3 buckets, and CloudFormation stacks

Install the CDK CLI globally:

npm install -g aws-cdk
cdk --version

Initialise project

Create a new directory and initialise a CDK Python project inside it:

mkdir my-cdk-app && cd my-cdk-app
cdk init app --language python

This scaffolds the project with a virtual environment, a sample stack, and a cdk.json config file.

Activate the virtual environment and install dependencies:

# macOS / Linux
source .venv/bin/activate

# Windows (PowerShell)
.venv\Scripts\activate.ps1

pip install -r requirements.txt

Directory structure

After initialisation the project looks like this:

my-cdk-app/
├── app.py                  # CDK app entry point
├── cdk.json                # CDK config and context
├── requirements.txt        # Python dependencies
├── requirements-dev.txt    # Dev dependencies (e.g. pytest)
├── .venv/                  # Python virtual environment
└── my_cdk_app/
    ├── __init__.py
    └── my_cdk_app_stack.py # Default stack

app.py is where the CDK app is defined and stacks are instantiated. my_cdk_app_stack.py is where resources are defined. The folder and class names match the project name passed to cdk init.

Bootstrap AWS account

CDK needs a one-time setup in each AWS account and region it deploys to. Bootstrapping creates an S3 bucket (for assets) and IAM roles (for deployments) in the account:

cdk bootstrap aws://123456789012/us-east-1

Replace 123456789012 with your account ID and us-east-1 with your target region. If credentials are already configured via AWS CLI or SSO, the account ID can be omitted:

cdk bootstrap

This only needs to run once per account/region combination. Re-running it is safe - it updates the bootstrap stack if needed.

Synth and deploy

CDK compiles Python into CloudFormation before deploying. Run synth to see the generated CloudFormation template:

cdk synth

This outputs a CloudFormation template to cdk.out/. Use it to verify what CDK will create before deploying.

Deploy the stack:

cdk deploy

CDK shows a diff of changes and prompts for confirmation before creating or modifying any IAM resources.

Verify

After deploying, the stack appears in CloudFormation in the AWS console. Check:

CloudFormation > Stacks > MyCdkAppStack - status should be CREATE_COMPLETE
S3 > Buckets - the CDK bootstrap bucket should be visible (cdk-hnb659fds-assets-...)

To tear down the stack:

cdk destroy

Notes

The bootstrap stack itself is a CloudFormation stack - it can be viewed and managed in the console under CDKToolkit.
Each AWS account + region combination needs its own bootstrap - e.g. deploying to both us-east-1 and ap-southeast-2 requires bootstrapping both.
cdk synth is useful for debugging - the generated CloudFormation is in cdk.out/ and can be inspected directly.
The .venv/ folder and cdk.out/ should be in .gitignore - they are added automatically by cdk init.

Install CloudWatch agent in EC2 Instance

Thu, 03 Apr 2025 00:00:00 +0000

EC2 does not send memory or disk metrics to CloudWatch by default - only CPU, network and status checks. The CloudWatch agent runs inside the instance and collects system-level metrics and logs directly.

Prerequisites

EC2 instance running (Amazon Linux 2 or Ubuntu)
SSH access to the instance - see Connect to AWS SSO and SSH into EC2 Instance
Terminal

Attach IAM role to the instance

The agent needs permission to write metrics and logs to CloudWatch. In the AWS console:

EC2 > Instances > select your instance
Actions > Security > Modify IAM role
Attach a role that has CloudWatchAgentServerPolicy managed policy

If no such role exists, create one:

IAM > Roles > Create role
Trusted entity: AWS service > EC2
Add permission: CloudWatchAgentServerPolicy
Name it (e.g. ec2-cloudwatch-agent-role) and create

Install the agent

SSH into the instance, then run the appropriate command for your OS.

Amazon Linux 2:

sudo yum install -y amazon-cloudwatch-agent

Ubuntu:

sudo apt-get install -y amazon-cloudwatch-agent

Configure the agent

Run the configuration wizard - it steps through metrics and log collection interactively:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

Alternatively, create a config file manually at /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "root"
  },
  "metrics": {
    "namespace": "CWAgent",
    "metrics_collected": {
      "mem": {
        "measurement": ["mem_used_percent"]
      },
      "disk": {
        "measurement": ["disk_used_percent"],
        "resources": ["/"]
      }
    }
  }
}

Collects memory and disk usage every 60 seconds under the CWAgent namespace.

Start the agent

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config \
  -m ec2 \
  -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json \
  -s

Verify

Check the agent is running:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Expected output:

{
  "status": "running",
  "starttime": "...",
  "version": "..."
}

In the AWS console, go to CloudWatch > Metrics > All metrics > CWAgent to confirm metrics are coming in. Give it 2-3 minutes after starting for metrics to show up.

Notes

If the instance had no IAM role before, restart the agent after attaching the role - it picks up credentials on start.
The wizard config is saved to /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json by default.
To collect logs, add a logs block to the config pointing to the log file paths.
Agent logs are at /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log - check here if metrics are not appearing.

Connect to AWS SSO and SSH into EC2 Instance

Mon, 17 Mar 2025 00:00:00 +0000

Two paths depending on your setup - follow one:

Without SSO: personal AWS account, direct IAM credentials
With SSO: enterprise/team setup with an SSO start URL (e.g. https://company.awsapps.com/start)

Prerequisites

Without SSO:

AWS CLI v2
AWS IAM user credentials (Access Key ID and Secret Access Key)
Terminal

With SSO:

AWS CLI v2
AWS SSO start URL and access
Terminal

Files you will need

~/
├── .aws/
│   ├── config
│   └── credentials
└── .ssh/
    ├── ec2-key.pem
    └── config

Path A: Without SSO

Create .aws folder if it doesn’t exist. Run from your home directory (~).

mkdir -p ~/.aws

Create ~/.aws/credentials.

[default]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_ACCESS_KEY

Create ~/.aws/config.

[default]
region = ap-southeast-2
output = json

Test your config.

aws sts get-caller-identity

Path B: With SSO

Add a profile block to ~/.aws/config for each environment (e.g. aws-dev, aws-prod):

[profile aws-prod]
sso_session = aws-prod
sso_account_id = 123456789123
sso_role_name = aws-prod-SystemAdmin
region = ap-southeast-4
output = json
credential_process = aws configure export-credentials --profile aws-prod

Sample .aws folder structure:

Setup SSH config

Download the .pem keypair when launching the EC2 instance and move it here.

mkdir -p ~/.ssh
mv ~/Downloads/ec2-prod.pem ~/.ssh/ec2-prod.pem
chmod 400 ~/.ssh/ec2-prod.pem

Basic SSH config (no SSO)

Add to ~/.ssh/config:

Host ec2-prod
    HostName 1.2.3.4             # Replace with your EC2's public IP
    User ec2-user                # Use 'ubuntu' for Ubuntu instances
    IdentityFile ~/.ssh/ec2-prod.pem

SSO ProxyCommand config

Use this if connecting via AWS SSM Session Manager with an SSO profile. Add to ~/.ssh/config (or a separate file under ~/.ssh/config.d/):

Host ec2-prod
  User ec2-user
  IdentityFile ~/.ssh/ec2-prod.pem
  ProxyCommand aws ssm start-session --target i-01xyz7659824a123q --profile aws-prod --document-name AWS-StartSSHSession --parameters portNumber=22

Sample .ssh folder structure:

Connect

If using SSO, log in first:

aws sso login --profile aws-prod

Then SSH in:

ssh ec2-prod

CICD on Bastab C

Infrastructure as Code with AWS CDK - Series Overview

Why CDK over CloudFormation for AWS-native workloads?

Why CDK over Terraform for AWS-native workloads?

Use case

The series

Chapter 1 - Project setup and bootstrapping

Chapter 2 - Managing configuration and context

Chapter 3 - Writing and sharing constructs

Chapter 4 - Testing CDK stacks

Chapter 5 - CI/CD with CDK

Notes

AWS Kiro CLI - Manage AWS Infrastructure from the Terminal

Install

Authenticate

Key commands

Infrastructure workflows

Notes

GitHub PR Fix: "Commits must have verified signatures" blocking PR merge

Problem

Root cause

Steps to fix

1. Identify unsigned commits

2. Re-sign all branch commits using filter-branch

3. Verify no unsigned commits remain

4. Force push

5. Update local tracking ref (if GitHub Desktop shows stale state)

Notes

AWS CDK - CI/CD Pipelines

Common flags for CI

GitHub Actions

Auth

Workflow

Jenkins

Auth

Jenkinsfile

Azure DevOps

Auth

Pipeline

AWS CodePipeline

Notes

AWS CDK - Testing CDK Stacks

How CDK testing works

Setup

Testing the construct

Testing the stack

Snapshot testing

Running tests

Notes

AWS CDK - Writing and Sharing Constructs

L1, L2, L3

The problem

Writing the construct

Using it in a stack

Reusing the construct

Using exposed attributes

Notes

AWS CDK - Managing Configuration and Context

Quick comparison

Local dev patterns

Static config (cdk.json)

Dynamic config

Production patterns

Secrets Manager

CI/CD context injection

Notes

AWS CDK - Project Setup and Bootstrapping

Prerequisites

Initialise project

Directory structure

Bootstrap AWS account

Synth and deploy

Verify

Notes

Install CloudWatch agent in EC2 Instance

Prerequisites

Attach IAM role to the instance

Install the agent

Configure the agent

Start the agent

Static config (`cdk.json`)