E2E Testing

Stoobly Scaffold E2E Testing - Questions & Answers

Stoobly scaffold provides first-class support for E2E testing frameworks like Playwright and Cypress, enabling you to record, mock, and validate API interactions during your end-to-end tests.

Plugin Support

Q: What E2E testing frameworks does Stoobly support?

A: Stoobly supports Playwright and Cypress through the --plugin option when creating an app.

Example:

# Create app with Playwright support
poetry run stoobly-agent scaffold app create my-app --plugin playwright

# Create app with Cypress support
poetry run stoobly-agent scaffold app create my-app --plugin cypress

# Create app with both plugins
poetry run stoobly-agent scaffold app create my-app --plugin playwright --plugin cypress

Q: What files are created when I use the --plugin option?

A: When you specify a plugin, Stoobly creates a custom Dockerfile and entrypoint script in the entrypoint/test/ directory specifically configured for that E2E framework.

Files created for Playwright:

  • .stoobly/services/entrypoint/test/.Dockerfile.playwright - Playwright-specific Docker image with browser dependencies

  • .stoobly/services/entrypoint/test/.entrypoint.sh - Entrypoint script with CA certificate setup

  • .stoobly/services/entrypoint/test/docker-compose.yml - Docker compose configuration

  • .stoobly/services/entrypoint/test/configure - Stoobly configuration script

  • .stoobly/services/entrypoint/test/init - Initialization script

Files created for Cypress:

  • .stoobly/services/entrypoint/test/.Dockerfile.cypress - Cypress-specific Docker image

  • .stoobly/services/entrypoint/test/docker-compose.yml - Docker compose configuration

  • .stoobly/services/entrypoint/test/configure - Stoobly configuration script

  • .stoobly/services/entrypoint/test/init - Initialization script

Initial Setup

Q: How do I scaffold an app for Playwright E2E testing?

A: Create an app with the --plugin playwright option to set up Playwright-specific configurations.

Example:

# Create app with Playwright
poetry run stoobly-agent scaffold app create my-e2e-tests \
  --app-dir-path ./my-project \
  --plugin playwright \
  --run-on docker

# Add frontend service
poetry run stoobly-agent scaffold service create frontend \
  --app-dir-path ./my-project \
  --hostname app.local \
  --upstream-hostname localhost \
  --upstream-port 3000 \
  --workflow test

# Add API service
poetry run stoobly-agent scaffold service create api \
  --app-dir-path ./my-project \
  --hostname api.local \
  --upstream-hostname api.production.com \
  --upstream-port 443 \
  --scheme https \
  --workflow test

Q: How do I scaffold an app for Cypress E2E testing?

A: Create an app with the --plugin cypress option to set up Cypress-specific configurations.

Example:

# Create app with Cypress
poetry run stoobly-agent scaffold app create my-cypress-tests \
  --app-dir-path ./my-project \
  --plugin cypress \
  --run-on docker

# Add services as needed
poetry run stoobly-agent scaffold service create app \
  --app-dir-path ./my-project \
  --hostname myapp.local \
  --workflow test

Configuration Files

Q: What files should I modify to add my Playwright tests?

A: After scaffolding, you need to modify the entrypoint service docker-compose.yml to run your tests, the configure script to set up Stoobly rules, and your Playwright configuration to use the proxy.

Example - Modify docker-compose.yml:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    build:
      context: .
      dockerfile: .Dockerfile.playwright
      args:
        PLAYWRIGHT_IMAGE: mcr.microsoft.com/playwright:v1.40.0-jammy
        USER_ID: ${USER_ID}
    working_dir: /home/stoobly/tests
    volumes:
      - ../../..:/home/stoobly/tests    # Mount your test directory
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
      - NO_PROXY=localhost,127.0.0.1
    profiles:
      - ${WORKFLOW_NAME}
    command: npx playwright test          # Your test command

Example - Modify configure script:

# .stoobly/services/entrypoint/test/configure
#!/bin/bash

# Rewrite production URLs to local services
poetry run stoobly-agent config rewrite set \
  --pattern "https://api.production.com/.*" \
  --method GET --method POST \
  --mode test \
  --hostname api.local

# Exclude third-party tracking
poetry run stoobly-agent config firewall set \
  --pattern "https://.*analytics.com/.*" \
  --method GET \
  --mode test \
  --action exclude

Example - Create Playwright config:

// playwright.config.ts (in your project root)
import { defineConfig } from '@playwright/test';

export default defineConfig({
  use: {
    proxy: {
      server: 'http://gateway:80',
    },
    ignoreHTTPSErrors: true,
  },
});

Q: What files should I modify to add my Cypress tests?

A: For Cypress, modify the entrypoint service docker-compose.yml and Cypress configuration to work with Stoobly's proxy.

Example - Modify docker-compose.yml:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.cypress:
    build:
      context: .
      dockerfile: .Dockerfile.cypress
      args:
        CYPRESS_IMAGE: cypress/included:13.6.0
        USER_ID: ${USER_ID}
    working_dir: /home/stoobly/e2e
    volumes:
      - ../../..:/home/stoobly/e2e        # Mount your test directory
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
    profiles:
      - ${WORKFLOW_NAME}
    command: cypress run                   # Your test command

Example - Create Cypress config:

// cypress.config.js (in your project root)
const { defineConfig } = require('cypress');

module.exports = defineConfig({
  e2e: {
    env: {
      HTTP_PROXY: 'http://gateway:80',
      HTTPS_PROXY: 'http://gateway:80',
    },
  },
});

Q: How do I add custom npm packages for my tests?

A: Modify the Dockerfile to install additional dependencies.

Example for Playwright:

# .stoobly/services/entrypoint/test/.Dockerfile.playwright
ARG PLAYWRIGHT_IMAGE
FROM ${PLAYWRIGHT_IMAGE}

# Install gosu and ca-certificates
RUN set -eux; \
    apt-get update; \
    apt-get install -y --no-install-recommends ca-certificates wget; \
    dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')"; \
    wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/1.16/gosu-$dpkgArch"; \
    chmod +x /usr/local/bin/gosu; \
    gosu --version;

# Install additional npm packages
RUN npm install -g axios dotenv @faker-js/faker

# ... rest of Dockerfile

Q: How do I pass environment variables to my tests?

A: Add environment variables in the docker-compose.yml file under the environment section.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
      - BASE_URL=http://app.local
      - API_URL=http://api.local
      - [email protected]
      - TEST_USER_PASSWORD=testpass123
      - NODE_ENV=test

Running E2E Tests

Q: How do I record E2E test traffic?

A: Use the record workflow to capture all HTTP requests made during your E2E tests.

Example:

# Start record workflow
make -f .stoobly/services/Makefile record

# Tests run automatically in entrypoint container
# All HTTP requests are recorded

# View recorded requests
poetry run stoobly-agent request list

# Create scenario from recorded requests
poetry run stoobly-agent scenario create "E2E User Flow"

# Stop workflow
make -f .stoobly/services/Makefile record/down

Q: How do I run E2E tests with mocked responses?

A: Use the mock workflow to run tests against recorded responses without hitting real APIs.

Example:

# Ensure you have recorded responses
poetry run stoobly-agent scenario list

# Start mock workflow
make -f .stoobly/services/Makefile mock

# Tests run against mocked responses
# No real API calls are made

# Stop workflow
make -f .stoobly/services/Makefile mock/down

Q: How do I run E2E tests with response validation?

A: Use the test workflow to validate that responses match expected results.

Example:

# Start test workflow
make -f .stoobly/services/Makefile test

# Tests run with response validation
# Stoobly compares actual vs expected responses

# View test results
make -f .stoobly/services/Makefile test/logs

# Stop workflow
make -f .stoobly/services/Makefile test/down

Q: How do I run specific test files or suites?

A: Modify the command in docker-compose.yml to target specific tests.

Example for Playwright:

# Run specific test file
command: npx playwright test tests/login.spec.ts

# Run tests with specific tag
command: npx playwright test --grep @smoke

# Run tests in headed mode for debugging
command: npx playwright test --headed

Example for Cypress:

# Run specific spec
command: cypress run --spec "cypress/e2e/login.cy.js"

# Run with specific browser
command: cypress run --browser chrome

# Run with video recording
command: cypress run --video

Debugging

Q: How do I view test output and logs?

A: Use the logs command to see test execution output.

Example:

# View logs while tests are running
make -f .stoobly/services/Makefile test/logs

# View logs for specific service
docker-compose -f .stoobly/services/entrypoint/test/docker-compose.yml logs -f

Q: How do I debug failing tests?

A: Enable headed mode and increase logging verbosity in the docker-compose.yml.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    environment:
      - DEBUG=pw:api                    # Playwright debug logs
      - PWDEBUG=1                       # Playwright debug mode
    command: npx playwright test --headed --debug

Q: How do I save test artifacts (screenshots, videos)?

A: Mount an artifacts directory to persist test outputs and configure your test framework to save artifacts.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    volumes:
      - ../../..:/home/stoobly/tests
      - ../../../test-results:/home/stoobly/tests/test-results  # Artifacts
// playwright.config.ts
export default defineConfig({
  use: {
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  outputDir: 'test-results',
});

Advanced Configuration

Q: How do I mount test fixtures and data?

A: Add volume mounts to share fixtures between host and container.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    volumes:
      - ../../..:/home/stoobly/tests           # Project root
      - ../../../test-data:/home/stoobly/data  # Test data
      - ../../../fixtures:/home/stoobly/fixtures # Fixtures

Q: How do I handle dynamic data in E2E tests?

A: Use Stoobly's rewrite rules to normalize dynamic values in the configure script.

Example:

# .stoobly/services/entrypoint/test/configure
#!/bin/bash

# Replace dynamic user IDs with fixed test ID
poetry run stoobly-agent config rewrite set \
  --pattern "https://api.local/users/.*" \
  --method GET \
  --mode test \
  --type response_param \
  --name "userId" \
  --value "test-user-123"

# Replace timestamps
poetry run stoobly-agent config rewrite set \
  --pattern "https://api.local/.*" \
  --method GET --method POST \
  --mode test \
  --type response_param \
  --name "timestamp" \
  --value "2024-01-01T00:00:00Z"

Best Practices

Q: Should I record once and mock for all subsequent runs?

A: Yes, this is the recommended approach for fast, reliable E2E tests that don't depend on external APIs.

Example:

# Step 1: Record once (against real APIs)
make -f .stoobly/services/Makefile record

# Step 2: Create scenario from recordings
poetry run stoobly-agent scenario create "User Journey - Login to Checkout"

# Step 3: Snapshot for version control
poetry run stoobly-agent scenario snapshot user-journey --decode
git add .stoobly/snapshots/ && git commit -m "Add E2E test snapshots"

# Step 4: All future runs use mocks (fast, no external dependencies)
make -f .stoobly/services/Makefile mock

Q: How do I organize E2E tests by feature?

A: Create separate services or custom workflows for different test suites.

Example:

# Create service for auth tests
poetry run stoobly-agent scaffold service create auth-tests \
  --app-dir-path ./my-project \
  --workflow test

# Create service for checkout tests
poetry run stoobly-agent scaffold service create checkout-tests \
  --app-dir-path ./my-project \
  --workflow test

# Run specific test suite
make -f .stoobly/services/Makefile test STOOBLY_WORKFLOW_SERVICE_OPTIONS="--service auth-tests"

Complete Example

Q: What's a complete example of setting up Playwright E2E tests?

A: Here's a full example from scaffold to running tests.

Step 1: Scaffold the app

# Create app with Playwright
poetry run stoobly-agent scaffold app create my-e2e-app \
  --app-dir-path ./my-project \
  --plugin playwright \
  --proxy-port 8080 \
  --ui-port 4200

# Add frontend service
poetry run stoobly-agent scaffold service create frontend \
  --app-dir-path ./my-project \
  --hostname app.local \
  --upstream-hostname localhost \
  --upstream-port 3000 \
  --workflow test

# Add API service
poetry run stoobly-agent scaffold service create api \
  --app-dir-path ./my-project \
  --hostname api.local \
  --upstream-hostname api.production.com \
  --scheme https \
  --port 443 \
  --workflow test

Step 2: Configure entrypoint docker-compose.yml

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright:
    build:
      context: .
      dockerfile: .Dockerfile.playwright
      args:
        PLAYWRIGHT_IMAGE: mcr.microsoft.com/playwright:v1.40.0-jammy
        USER_ID: ${USER_ID}
    working_dir: /home/stoobly/app
    volumes:
      - ../../..:/home/stoobly/app
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
      - BASE_URL=http://app.local
      - API_URL=http://api.local
    profiles:
      - ${WORKFLOW_NAME}
    command: npx playwright test

Step 3: Configure Stoobly rules

# .stoobly/services/entrypoint/test/configure
#!/bin/bash

# Rewrite API calls
poetry run stoobly-agent config rewrite set \
  --pattern "https://api.production.com/.*" \
  --method GET --method POST --method PUT --method DELETE \
  --mode test \
  --hostname api.local

# Exclude analytics
poetry run stoobly-agent config firewall set \
  --pattern "https://.*analytics.com/.*" \
  --method GET --method POST \
  --mode test \
  --action exclude

Step 4: Create Playwright config

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  use: {
    baseURL: process.env.BASE_URL || 'http://app.local',
    proxy: {
      server: 'http://gateway:80',
    },
    ignoreHTTPSErrors: true,
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  outputDir: 'test-results',
});

Step 5: Create test

// tests/login.spec.ts
import { test, expect } from '@playwright/test';

test('user can login', async ({ page }) => {
  await page.goto('/login');
  
  await page.fill('[name="email"]', '[email protected]');
  await page.fill('[name="password"]', 'password123');
  await page.click('button[type="submit"]');
  
  await expect(page).toHaveURL('/dashboard');
  await expect(page.locator('h1')).toContainText('Welcome');
});

Step 6: Run tests

# Run with recording
make -f .stoobly/services/Makefile record

# Run with mocking
make -f .stoobly/services/Makefile mock

# Run with validation
make -f .stoobly/services/Makefile test

Quick Reference

Key files to modify for E2E testing:

.stoobly/services/entrypoint/test/
├── docker-compose.yml          # Add your test execution command
├── .Dockerfile.playwright      # Add dependencies (Playwright)
├── .Dockerfile.cypress         # Add dependencies (Cypress)
├── configure                   # Set Stoobly rules
└── init                        # Add initialization logic

Your project root:
├── playwright.config.ts        # Configure Playwright
├── cypress.config.js           # Configure Cypress
├── tests/                      # Your test files
└── package.json                # Test dependencies

Common commands:

# Create app with E2E plugin
poetry run stoobly-agent scaffold app create my-app --plugin playwright
poetry run stoobly-agent scaffold app create my-app --plugin cypress

# Run E2E tests
make -f .stoobly/services/Makefile record    # Record API calls
make -f .stoobly/services/Makefile mock      # Run with mocks
make -f .stoobly/services/Makefile test      # Run with validation

# View logs
make -f .stoobly/services/Makefile test/logs

# Stop workflow
make -f .stoobly/services/Makefile test/down
PreviousCustomizationNextScenario

Last updated

Was this helpful?