Serverless CD Workflow Skill

Overview

The Serverless-CD workflow is an opinionated reusable workflow that deploys serverless applications to AWS Lambda or Azure Functions based on Terraform infrastructure-as-code. This workflow primarily focuses on deploying Lambda functions with code integration from S3 buckets or Artifactory, managing infrastructure state, and supporting multi-cloud deployments.

Workflow Reference

Repository: dojo360/pipelines-workflows
Workflow: .github/workflows/serverless-cd.yml
Version: v2.0.0 (stable) or @beta (latest)
Documentation: web/serverless-cd/index.md

Key Features

• Multi-Cloud Support: Deploy to AWS Lambda or Azure Functions based on cloud-type configuration
• Terraform-Based: Infrastructure managed via Terraform with support for multiple backend types
• OIDC Authentication: Secure keyless authentication for AWS (awsOptum) and Azure (azureOptum)
• S3 Integration: Direct integration with S3 buckets for Lambda function code storage
• Artifact Management: Support for artifact retrieval from Artifactory with automatic promotion
• Secrets Management: Integration with PRM (Password Repository Manager), Volcan, and Terraform Enterprise
• Environment Validation: Pull request validation with Terraform plan comments
• E2E Testing: Optional end-to-end test execution after successful deployment

Prerequisites

1. Metadata API: Product must be onboarded to Dojo360 Metadata API OR have a local metadata file
2. OIDC Configuration: Required for awsOptum and azureOptum cloud types:
   • AWS OIDC Setup
   • Azure OIDC Setup
3. Artifact Repository: Artifacts and artifact repository information required for code deployments
4. Terraform Modules: Must use Dojo360 AWS Modules for serverless resources

Requirements

• Terraform: ~> 1.9.x
• AWS Provider: ~> 5.xx (for AWS operations)
• AzureRM Provider: ~> 3.xx (for Azure operations)
• GCP Provider: ~> 6.xx (for GCP operations)

Required Inputs

Input	Type	Description
aide-id	string	AIDE ID for fetching team metadata
artifact-name	string	Name of the artifact containing serverless function code
cloud-type	string	Cloud provider for deployment. Options: awsOptum, awsChc20, azureOptum. See Supported Cloud Types
domain	string	Domain name for metadata lookup
environment	string	Target environment (dev, qa, cert, prod). Defines approval requirements
team-name	string	Team name for metadata lookup

Common Optional Inputs

Input	Type	Default	Description
artifact-environment	string	''	Artifact source environment. Options: RELEASE, CERTIFIED. Only used for default/tag/efix branches
artifact-repo-path	string	''	Path relative to repository where artifacts are stored
artifact-repository	string	''	Repository where artifacts are stored
artifact-s3-bucket-kms-key-id	string	''	KMS key ID for S3 server-side encryption (AWS)
comment-on-pr	boolean	false	Post Terraform plan output as PR comment
e2e-tests-enabled	boolean	false	Execute end-to-end tests after deployment
e2e-workflow-file	string	''	Workflow file to execute for E2E tests
e2e-workflow-inputs	string (JSON)	'{}'	JSON input overrides for E2E workflow
jfrog-project-key	string	''	JFrog project key for SaaS Artifactory
jfrog-release-bundle-name	string	''	Release bundle name for promotion
jfrog-release-bundle-version	string	''	Release bundle version for promotion
pr-number	string	''	PR number for comments (required if comment-on-pr is true)
prm-base-url	string	https://prm.optum.com	PRM instance base URL
ref	string	HEAD	Branch, tag, or SHA to checkout
remote-state-file-name	string	''	Remote Terraform state filename
run-plan-only	boolean	false	Only run terraform plan without apply
runner-labels	string	''	Comma-separated custom runner labels
terraform-directory	string	.	Directory path containing Terraform code
terraform-logging	string	off	Terraform logging level
terraform-prm-secrets	string	''	Comma-separated list of PRM secrets to map to tfvars
terraform-vars-files	string	''	Comma-separated list of tfvars files
terraform-vars-values	string	''	Comma-separated key=value pairs for Terraform variables

Terraform State Management

The workflow supports multiple backend types:

Azure Backend (Default)

with:
  # Azure backend is default
  azurerm-backend-resource-group-name: "terraform-state-rg"
  azurerm-backend-storage-account-name: "tfstatestorage"
  azurerm-backend-container-name: "tfstate"
  azurerm-backend-key: "serverless.tfstate"

AWS S3 Backend

with:
  backend-type: "s3"
  aws-s3-bucket-name: "my-terraform-state-bucket"
  aws-s3-region: "us-east-1"
  aws-s3-key: "serverless/terraform.tfstate"

GCS Backend

with:
  backend-type: "gcs"
  gcs-bucket-name: "my-terraform-state-bucket"
  gcs-prefix: "serverless"

Secrets Management

PRM (Password Repository Manager)

with:
  terraform-prm-secrets: "DB_PASSWORD,API_KEY,JWT_SECRET"
  prm-base-url: "https://prm.optum.com"

Volcan Secrets

with:
  volcan-secrets-file: "secrets/prod.yaml"

Terraform Enterprise Secrets

Configure via Terraform Enterprise workspace variables.

Required Workflow Permissions

permissions:
  actions: read
  contents: write
  id-token: write  # Required for OIDC authentication
  pull-requests: read
  security-events: write

Required Secrets

• GH_TOKEN: GitHub token for workflow operations
• REPO_USERNAME: (Optional) Artifactory username
• REPO_PASSWORD: (Optional) Artifactory password

Usage Examples

1. Basic Lambda Deployment (AWS)

name: Deploy Serverless Function

on:
  push:
    branches:
      - main
  workflow_dispatch:
    inputs:
      environment:
        description: 'Target environment'
        required: true
        type: choice
        options:
          - dev
          - qa
          - cert
          - prod

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: read
  security-events: write

jobs:
  deploy:
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      # Required inputs
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: ${{ inputs.environment || 'dev' }}
      team-name: "<change me>"
      artifact-name: "my-lambda-function"
      
      # Artifact configuration
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-np-loc/lambda-functions"
      jfrog-project-key: "<change me>"
      
      # Terraform configuration
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "${{ inputs.environment || 'dev' }}.tfvars"
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

2. Azure Functions Deployment

name: Deploy Azure Function

on:
  push:
    branches:
      - main

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: read
  security-events: write

jobs:
  deploy:
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      # Required inputs
      aide-id: "<change me>"
      cloud-type: "azureOptum"
      domain: "<change me>"
      environment: "dev"
      team-name: "<change me>"
      artifact-name: "my-azure-function"
      
      # Azure-specific configuration
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-np-loc/azure-functions"
      jfrog-project-key: "<change me>"
      
      # Terraform configuration
      terraform-directory: "terraform/azure-function"
      terraform-vars-files: "dev.tfvars"
      
      # State management
      azurerm-backend-resource-group-name: "terraform-state-rg"
      azurerm-backend-storage-account-name: "tfstatestorage"
      azurerm-backend-container-name: "tfstate"
      azurerm-backend-key: "azure-function.tfstate"
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

3. PR Validation with Terraform Plan

name: Validate Serverless Changes

on:
  pull_request:
    branches:
      - main
    paths:
      - 'terraform/**'
      - 'src/**'

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: write  # Required for PR comments
  security-events: write

jobs:
  validate:
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: "dev"
      team-name: "<change me>"
      artifact-name: "my-lambda-function-pr-${{ github.event.pull_request.number }}"
      
      # Plan-only mode for PR validation
      run-plan-only: true
      comment-on-pr: true
      pr-number: ${{ github.event.pull_request.number }}
      
      # Terraform configuration
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "dev.tfvars"
      terraform-logging: "info"  # Enable detailed logging
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

4. Multi-Environment Deployment with Artifact Promotion

name: Serverless CD Pipeline

on:
  push:
    branches:
      - main
      - develop
      - 'release/**'

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: read
  security-events: write

jobs:
  # Deploy to Dev from develop branch
  deploy-dev:
    if: github.ref == 'refs/heads/develop'
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: "dev"
      team-name: "<change me>"
      artifact-name: "my-lambda-${{ github.sha }}"
      
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-np-loc/lambda"
      jfrog-project-key: "<change me>"
      
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "dev.tfvars"
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}
  
  # Deploy to QA from main branch with artifact promotion
  deploy-qa:
    if: github.ref == 'refs/heads/main'
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: "qa"
      team-name: "<change me>"
      artifact-name: "my-lambda-${{ github.sha }}"
      
      # Artifact promotion to RELEASE
      artifact-environment: "RELEASE"
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-release-loc/lambda"
      jfrog-project-key: "<change me>"
      
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "qa.tfvars"
      
      # Enable E2E tests after deployment
      e2e-tests-enabled: true
      e2e-workflow-file: ".github/workflows/e2e-tests.yml"
      e2e-workflow-inputs: |
        {
          "environment": "qa",
          "api_endpoint": "https://lambda-qa.example.com"
        }
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

5. Deployment with PRM Secrets

name: Deploy with Secrets

on:
  workflow_dispatch:
    inputs:
      environment:
        required: true
        type: choice
        options:
          - cert
          - prod

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: read
  security-events: write

jobs:
  deploy:
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: ${{ inputs.environment }}
      team-name: "<change me>"
      artifact-name: "my-lambda-v${{ github.run_number }}"
      
      artifact-environment: "CERTIFIED"
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-release-loc/lambda"
      jfrog-project-key: "<change me>"
      
      # Terraform configuration with secrets
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "${{ inputs.environment }}.tfvars,common.tfvars"
      terraform-prm-secrets: "DB_PASSWORD,API_KEY,JWT_SECRET,ENCRYPTION_KEY"
      prm-base-url: "https://prm.optum.com"
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

6. CI/CD Chain with Serverless Build

name: Build and Deploy Lambda

on:
  push:
    branches:
      - main

permissions:
  actions: read
  contents: write
  id-token: write
  pull-requests: write
  security-events: write
  checks: write

jobs:
  # CI - Build serverless artifact
  build:
    uses: uhg-pipelines/ci-workflows/.github/workflows/python-ci.yml@v2
    with:
      python-version: "3.11"
      jfrog-project-key: "<change me>"
      enable-sonar: true
      
      # Serverless artifact configuration
      generic-package: true
      generic-artifact-repo-path: "<change me>-generic-np-loc/lambda"
      source-folder: "./src"
      
  # CD - Deploy Lambda function
  deploy:
    needs: build
    uses: dojo360/pipelines-workflows/.github/workflows/[email protected]
    with:
      aide-id: "<change me>"
      cloud-type: "awsOptum"
      domain: "<change me>"
      environment: "dev"
      team-name: "<change me>"
      
      # Use artifact from CI build
      artifact-name: ${{ needs.build.outputs.uploaded-artifact-name }}
      artifact-repository: "centraluhg.jfrog.io"
      artifact-repo-path: "<change me>-generic-np-loc/lambda"
      jfrog-project-key: "<change me>"
      
      terraform-directory: "terraform/lambda"
      terraform-vars-files: "dev.tfvars"
      
    secrets:
      GH_TOKEN: ${{ secrets.GH_TOKEN }}

Cloud-Type Configurations

awsOptum (OIDC Authentication)

• Uses OIDC for secure, keyless authentication
• Requires OIDC configuration in AWS account
• Runner selection automatic based on metadata
• Terraform provider mirror automatic

awsChc20 (Instance Profile)

• Uses EC2 instance profile authentication
• Runner credentials come from instance role
• Terraform provider mirror automatic

azureOptum (OIDC Authentication)

• Uses OIDC for Azure authentication
• Requires OIDC configuration in Azure subscription
• Azure CLI authentication automatic
• Terraform AzureRM backend support

Terraform Output Requirements

Your Terraform code must output specific values for the workflow to function properly:

# For AWS Lambda deployments
output "lambda_function_name" {
  description = "Name of the Lambda function"
  value       = aws_lambda_function.main.function_name
}

output "lambda_s3_bucket" {
  description = "S3 bucket for Lambda code"
  value       = aws_s3_bucket.lambda_code.id
}

output "lambda_s3_key" {
  description = "S3 key for Lambda code"
  value       = "lambda/${var.artifact_name}.zip"
}

# For Azure Functions deployments
output "function_app_name" {
  description = "Name of the Azure Function App"
  value       = azurerm_function_app.main.name
}

output "resource_group_name" {
  description = "Name of the resource group"
  value       = azurerm_resource_group.main.name
}

Artifact Repository Patterns

Non-Production Deployments (Feature Branches)

• Default Repository: <project-key>-generic-np-loc
• Example: harmony-generic-np-loc/lambda-functions/my-function.zip

Production Deployments (Main/Tag/Efix Branches)

• RELEASE Repository: <project-key>-generic-release-loc
• CERTIFIED Repository: <project-key>-generic-certified-loc
• Controlled by: artifact-environment input

Artifact Promotion Flow

1. Feature Branch → generic-np-loc
2. Main Branch → Promotes to generic-release-loc if artifact-environment: RELEASE
3. Release Tag → Promotes to generic-certified-loc if artifact-environment: CERTIFIED

Best Practices

1. Version Management

• Use semantic versioning for artifacts
• Tag releases consistently
• Reference specific workflow versions (@v2.0.0) in production

2. Terraform Best Practices

• Pin provider versions in your Terraform code
• Use separate tfvars files per environment
• Implement proper state locking
• Use remote state backends for team collaboration

3. Security

• Always use OIDC authentication when available
• Store secrets in PRM or secure vault
• Never commit credentials to repository
• Use artifact-environment for promotion controls

4. Testing

• Implement PR validation with plan-only mode
• Use E2E testing after deployments
• Validate in lower environments first

5. Monitoring & Observability

• Configure Lambda/Function logging
• Set up CloudWatch or Application Insights
• Implement proper error handling
• Use structured logging

6. Artifact Management

• Use consistent naming conventions
• Include version/SHA in artifact names
• Implement proper retention policies
• Document artifact structure

7. Environment Strategy

• Maintain separate configurations per environment
• Use environment protection rules
• Implement approval gates for production
• Test promotion workflows in non-prod first

Troubleshooting

1. Artifact Not Found

Symptom: Workflow fails with "artifact not found" error

Solutions:

• Verify artifact name matches exactly
• Check artifact-repo-path is correct
• Ensure jfrog-project-key is configured
• Verify artifact exists in expected repository
• Check if artifact promotion is needed

2. Terraform State Lock

Symptom: "Error acquiring the state lock"

Solutions:

• Check for concurrent workflow runs
• Verify state backend configuration
• Use terraform-ops workflow to force unlock if needed
• Implement proper run concurrency controls

3. OIDC Authentication Failures

Symptom: "Failed to authenticate with OIDC provider"

Solutions:

• Verify OIDC is configured in AWS/Azure
• Check IAM role trust relationship
• Ensure id-token: write permission is set
• Verify cloud-type matches OIDC setup

4. S3 Bucket Access Errors

Symptom: "Access Denied" when accessing S3 bucket

Solutions:

• Verify IAM role has S3 permissions
• Check S3 bucket policy
• Ensure KMS key permissions if encrypted
• Verify bucket exists in correct region

5. Environment Approval Timeout

Symptom: Workflow waits indefinitely for approval

Solutions:

• Verify environment has required reviewers configured
• Check GitHub environment protection rules
• Ensure reviewers have proper permissions
• Review pending deployment approvals

Support & Documentation

• Workflow Documentation: Serverless-CD Index
• Sample Applications: Serverless Compute Samples
• CloudBricks Guide: Working with CloudBricks
• Supported Cloud Types: Cloud Types Documentation
• CI/CD Examples: CI Workflows Demos
• Dojo360 Modules: AWS Modules

---

Note: This workflow is marked as "Emerging" PADU (Platform Adoption Degree of Use). Features and inputs may evolve. Always refer to the latest documentation and pin to specific workflow versions for production use.