Implement Specific Testing Layer

You are implementing a specific testing layer (unit, functional, integration, or performance) in an existing codebase. Your goal is to set up the infrastructure, write tests, integrate with CI/CD, and establish best practices.

Implementation Process

1. Define Scope and Requirements

Test Layer: [unit/functional/integration/performance] Language/Framework: [JavaScript/Python/Go/etc.] Test Runner: [Jest/Mocha/pytest/Go test/etc.] Target Coverage: [80% lines/branches/etc.] Runtime Budget: [< X seconds/minutes]

Example:

Test Layer: Integration Language: JavaScript (Node.js) Test Runner: Jest with supertest Target Coverage: 70% of API endpoints Runtime Budget: <10 minutes

2. Set Up Test Infrastructure

For Unit Tests

# Install test framework
npm install --save-dev jest @types/jest

# Configure jest.config.js
{
  "testEnvironment": "node",
  "testMatch": ["**/__tests__/**/*.test.js"],
  "coverageDirectory": "coverage",
  "collectCoverageFrom": ["src/**/*.js"]
}

# Add npm scripts
"scripts": {
  "test": "jest",
  "test:watch": "jest --watch",
  "test:coverage": "jest --coverage"
}

For Integration Tests

# Install integration test dependencies
npm install --save-dev supertest docker-compose

# Create docker-compose.test.yml for test environment
version: '3.8'
services:
  test-db:
    image: postgres:15
    environment:
      POSTGRES_DB: test_db
      POSTGRES_USER: test_user
      POSTGRES_PASSWORD: test_pass
    ports:
      - "5432:5432"

# Create integration test setup
tests/integration/setup.js
tests/integration/teardown.js

For Performance Tests

# Install performance test tools
npm install --save-dev k6

# Create k6 test script
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },  // Ramp up
    { duration: '1m', target: 100 },  // Sustained load
    { duration: '10s', target: 0 },   // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'], // 95% of requests < 500ms
  },
};

export default function () {
  const res = http.get('http://localhost:3000/api/assets');
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
  sleep(1);
}

3. Write Test Examples

Unit Test Example

// tests/unit/asset-builder.test.js
import { buildAssetCatalog } from '../src/builders/asset-builder';

describe('Asset Builder', () => {
  it('should build catalog from asset files', () => {
    const assets = buildAssetCatalog('./fixtures/test-assets');

    expect(assets).toHaveLength(3);
    expect(assets[0]).toHaveProperty('id');
    expect(assets[0]).toHaveProperty('kind');
  });

  it('should validate asset metadata', () => {
    const invalidAsset = { id: 'test' }; // Missing required fields

    expect(() => validateAsset(invalidAsset)).toThrow('Missing required field: kind');
  });
});

Integration Test Example

// tests/integration/mcp-server.test.js
import request from 'supertest';
import { startServer } from '../src/mcp/server';

describe('MCP Server Integration', () => {
  let server;

  beforeAll(async () => {
    server = await startServer();
  });

  afterAll(async () => {
    await server.close();
  });

  it('should list available assets', async () => {
    const response = await request(server)
      .post('/rpc')
      .send({
        jsonrpc: '2.0',
        id: 1,
        method: 'tools/call',
        params: { name: 'list_assets', arguments: { kind: 'prompt' } },
      });

    expect(response.status).toBe(200);
    expect(response.body.result).toHaveProperty('assets');
    expect(response.body.result.assets.length).toBeGreaterThan(0);
  });
});

Performance Test Example

// tests/performance/catalog-load.k6.js
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  vus: 50,
  duration: '30s',
  thresholds: {
    http_req_duration: ['p(95)<200'], // 95% < 200ms
    http_req_failed: ['rate<0.01'],   // <1% failures
  },
};

export default function () {
  const res = http.get('http://localhost:8080/api/catalog');

  check(res, {
    'catalog loads successfully': (r) => r.status === 200,
    'response time acceptable': (r) => r.timings.duration < 300,
    'contains assets': (r) => JSON.parse(r.body).assets.length > 0,
  });
}

4. Integrate with CI/CD

GitHub Actions Workflow

name: Testing Pipeline

on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  unit-tests:
    name: Unit Tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run unit tests
        run: npm test -- --coverage

      - name: Upload coverage
        uses: codecov/codecov-action@v3

  integration-tests:
    name: Integration Tests
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_DB: test_db
          POSTGRES_USER: test_user
          POSTGRES_PASSWORD: test_pass
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4

      - name: Install dependencies
        run: npm ci

      - name: Run integration tests
        run: npm run test:integration
        env:
          DATABASE_URL: postgres://test_user:test_pass@localhost:5432/test_db  # pragma: allowlist secret

  performance-tests:
    name: Performance Tests
    runs-on: ubuntu-latest
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'

    steps:
      - uses: actions/checkout@v4

      - name: Install k6
        run: |
          sudo gpg -k
          sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
          echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
          sudo apt-get update
          sudo apt-get install k6

      - name: Run performance tests
        run: k6 run tests/performance/catalog-load.k6.js

5. Establish Test Organization

tests/
├── unit/
│   ├── builders/
│   │   ├── asset-builder.test.js
│   │   └── catalog-builder.test.js
│   ├── validators/
│   │   ├── schema-validator.test.js
│   │   └── metadata-validator.test.js
│   └── utils/
│       └── file-utils.test.js
├── integration/
│   ├── mcp-server.test.js
│   ├── vscode-extension.test.js
│   └── setup.js
├── performance/
│   ├── catalog-load.k6.js
│   ├── mcp-response.k6.js
│   └── asset-search.k6.js
├── fixtures/
│   ├── test-assets/
│   └── mock-data.json
└── helpers/
    ├── test-utils.js
    └── mock-server.js

6. Write Testing Documentation

Create docs/testing-guide.md:

# Testing Guide

## Running Tests

### Unit Tests
npm test                    # Run all unit tests
npm test -- --watch         # Watch mode
npm test -- --coverage      # With coverage report

### Integration Tests
npm run test:integration    # Requires Docker
docker-compose -f docker-compose.test.yml up -d
npm run test:integration
docker-compose -f docker-compose.test.yml down

### Performance Tests
npm run test:performance    # Requires k6
k6 run tests/performance/catalog-load.k6.js

## Writing Tests

### Unit Test Guidelines
- Test one thing per test
- Use descriptive test names
- Mock external dependencies
- Aim for 80% coverage

### Integration Test Guidelines
- Test real component interactions
- Use test fixtures for data
- Clean up after each test
- Keep tests isolated

### Performance Test Guidelines
- Define clear thresholds
- Use realistic load patterns
- Test under various conditions
- Monitor resource usage

## Test Fixtures

Test data is in `tests/fixtures/`. To add new fixtures:
1. Create fixture file
2. Import in test
3. Clean up after use

7. Add Quality Gates

Coverage Requirements

// jest.config.js
module.exports = {
  coverageThreshold: {
    global: {
      branches: 70,
      functions: 80,
      lines: 80,
      statements: 80,
    },
  },
};

Performance Thresholds

// tests/performance/thresholds.js
export const performanceThresholds = {
  http_req_duration: ['p(95)<500', 'p(99)<1000'],
  http_req_failed: ['rate<0.01'],
  http_reqs: ['rate>10'],
};

8. Monitor and Maintain

Test Health Metrics:

• Test execution time (trending)
• Flaky test rate (<2% target)
• Coverage percentage (trending)
• Test count vs codebase size ratio

Regular Maintenance:

• Weekly: Review flaky tests, update fixtures
• Monthly: Review slow tests, optimize test data
• Quarterly: Audit test coverage, refactor brittle tests

Output Format

Your implementation should include:

1. Test Infrastructure Setup

   • Dependencies installed (package.json updates)
   • Test configuration files (jest.config.js, etc.)
   • Docker compose for test environments (if needed)

2. Test Files

   • Example tests demonstrating patterns
   • Test helpers and utilities
   • Fixtures and mock data

3. CI/CD Integration

   • GitHub Actions workflow updates
   • Pre-commit hooks (if applicable)
   • Quality gates configuration

4. Documentation

   • Testing guide (docs/testing-guide.md)
   • Contributing guidelines update
   • README updates

5. Monitoring Setup

   • Test metrics dashboard (optional)
   • Alerting for test failures
   • Coverage trending

Best Practices

• Fast Feedback - Keep unit tests under 1 second each
• Isolation - Tests should not depend on each other
• Clarity - Test names should describe expected behavior
• Maintainability - Avoid brittle tests that break on refactoring
• Coverage - Aim for high coverage, but prioritize critical paths

Anti-Patterns to Avoid

❌ Testing implementation details - Test behavior, not internals ❌ Flaky tests - Fix or quarantine immediately ❌ Slow tests - Optimize or move to performance layer ❌ Copy-paste tests - Use test helpers and fixtures ❌ No test maintenance - Tests need refactoring too

Success Criteria

Implementation is complete when:

• All infrastructure is set up and working
• Example tests pass in CI/CD
• Documentation is complete
• Team can write and run tests independently
• Quality gates are enforced
• Test health is monitored