Customization

Stoobly Scaffold Structure - Questions & Answers

After scaffolding an app and services, Stoobly creates a comprehensive directory structure in .stoobly/services/ with workflow-specific configurations, Docker compose files, and helper scripts.

---

Understanding Scaffold Structure

Q: Where does the scaffold get created?

A: The scaffold is created in the .stoobly/services/ directory within your application directory.

Example:

# Create app
stoobly-agent scaffold app create my-app --app-dir-path ./my-project

# Scaffold structure created at:
# ./my-project/.stoobly/services/

Q: What is the overall structure of a scaffolded app?

A: The scaffold contains core services (build, entrypoint, gateway, stoobly-ui), user-defined services, a Makefile, and workflow-specific configurations.

Example:

.stoobly/services/
├── Makefile                    # Main Makefile for workflow commands
├── build/                      # Build service (Docker only)
│   ├── mock/
│   ├── record/
│   └── test/
├── entrypoint/                 # Entrypoint service (Docker only)
│   ├── mock/
│   ├── record/
│   └── test/
├── gateway/                    # Gateway service (Docker only)
│   ├── mock/
│   ├── record/
│   └── test/
├── stoobly-ui/                 # Stoobly UI service
│   ├── exec/
│   ├── mock/
│   └── record/
└── your-service/               # Your custom services
    ├── mock/
    ├── record/
    └── test/

---

Docker vs Local Runtime Differences

Q: What's the difference between --run-on docker and --run-on local?

A: Docker runtime creates additional core services (build, entrypoint, gateway) for containerized execution, while local runtime creates a simpler structure for native execution.

Example:

# Docker runtime (default) - Full container orchestration
stoobly-agent scaffold app create my-app --run-on docker

# Creates:
# - build/ (Docker image building)
# - entrypoint/ (Container entry point)
# - gateway/ (Traefik reverse proxy)
# - stoobly-ui/ (UI service)
# - your-services/

# Local runtime - Simplified structure
stoobly-agent scaffold app create my-app --run-on local

# Creates:
# - stoobly-ui/ (UI service)
# - your-services/ (only user services)
# No build/, entrypoint/, or gateway/ directories

Q: Which files are created for Docker runtime only?

A: Docker runtime creates build/, entrypoint/, and gateway/ service directories with Docker Compose configurations.

Example:

# Docker-only services:
.stoobly/services/
├── build/                      # Docker only
├── entrypoint/                 # Docker only
├── gateway/                    # Docker only
└── stoobly-ui/                 # Both Docker and local

Q: Which files are created for both Docker and local runtime?

A: Both runtimes create the Makefile, stoobly-ui service, and user-defined service directories.

Example:

# Common to both:
.stoobly/services/
├── Makefile                    # Both
├── stoobly-ui/                 # Both
└── your-service/               # Both

---

Understanding the Entrypoint Service

Q: What is the entrypoint service?

A: The entrypoint service is the main container that runs your application or test code within the Docker network. It's where your actual service or test scripts execute.

Example:

# Entrypoint service structure
.stoobly/services/entrypoint/
├── mock/
│   ├── configure               # Pre-run configuration script
│   ├── docker-compose.yml      # Service definition
│   └── init                    # Initialization script
├── record/
│   ├── configure
│   ├── docker-compose.yml
│   └── init
└── test/
    ├── configure
    ├── docker-compose.yml
    └── init

Q: What is the purpose of the entrypoint service?

A: The entrypoint service provides a containerized environment where your application runs and makes HTTP requests that get intercepted by Stoobly's proxy through the gateway.

Example:

# Workflow flow:
# 1. Entrypoint container starts
# 2. Your app/tests run inside entrypoint
# 3. HTTP requests go through gateway (Traefik)
# 4. Gateway routes to Stoobly proxy
# 5. Stoobly records/mocks/tests the requests

Q: When is the entrypoint service used?

A: The entrypoint service is used in Docker runtime workflows to run your application code, test suites, or any process that makes HTTP requests you want to intercept.

Example:

# Start mock workflow
make -f .stoobly/services/Makefile mock

# Entrypoint container runs your app
# App makes requests → Gateway → Stoobly proxy → Mocked responses

Q: How do I customize the entrypoint service?

A: Edit the docker-compose.yml file in the entrypoint workflow directory to add your application container, environment variables, or volumes.

Example:

# .stoobly/services/entrypoint/mock/docker-compose.yml
services:
  entrypoint.my-app:
    image: my-app:latest
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
    profiles:
      - ${WORKFLOW_NAME}
    command: npm test

Q: What's the difference between configure and init scripts?

A: The configure script runs before services start to set up Stoobly configuration, while init scripts run during container initialization.

Example:

# configure script - Runs first
# .stoobly/services/entrypoint/mock/configure
#!/bin/bash
# Add custom Stoobly configuration here
stoobly-agent config rewrite set --pattern "..." --hostname localhost

# init script - Runs during container startup
# Custom initialization logic

---

Understanding Other Core Services

Q: What is the build service?

A: The build service creates the Docker image for Stoobly agent with your application's context, ensuring consistent environments across workflows.

Example:

.stoobly/services/build/
├── mock/
│   ├── configure
│   ├── docker-compose.yml      # Builds stoobly image
│   └── init
└── ...

Q: What is the gateway service?

A: The gateway service runs Traefik reverse proxy that routes HTTP traffic from your services through Stoobly's proxy for interception.

Example:

.stoobly/services/gateway/
├── mock/
│   └── docker-compose.yml      # Traefik configuration
└── ...

# Gateway routes:
# your-service:80 → gateway:80 → stoobly-proxy:8080 → upstream

Q: What is the stoobly-ui service?

A: The stoobly-ui service provides the web interface for managing requests, scenarios, and configuration. It's available in both Docker and local runtimes.

Example:

.stoobly/services/stoobly-ui/
├── exec/                       # For CLI execution
├── mock/                       # UI for mock workflow
└── record/                     # UI for record workflow

# Access UI at: http://localhost:4200

---

Service-Specific Directories

Q: What files are created for each user-defined service?

A: Each service gets workflow directories (mock, record, test) with docker-compose.yml, configure script, init script, lifecycle hooks, fixtures, and public directory.

Example:

.stoobly/services/my-service/
├── mock/
│   ├── configure               # Configuration script
│   ├── docker-compose.yml      # Service definition
│   ├── fixtures.yml            # Response fixtures
│   ├── init                    # Init script
│   ├── lifecycle_hooks.py      # Custom hooks
│   └── public/                 # Static files
├── record/
│   ├── configure
│   ├── docker-compose.yml
│   ├── init
│   └── lifecycle_hooks.py
└── test/
    ├── configure
    ├── docker-compose.yml
    ├── fixtures.yml
    ├── init
    ├── lifecycle_hooks.py
    └── public/

Q: What is the purpose of the configure script?

A: The configure script sets up Stoobly configuration (rewrite rules, match rules, firewall rules) before the workflow starts.

Example:

# .stoobly/services/my-service/mock/configure
#!/bin/bash

# Add rewrite rules
stoobly-agent config rewrite set \
  --pattern "https://api.production.com/.*" \
  --method GET --method POST \
  --mode mock \
  --hostname localhost:8080

# Add firewall rules
stoobly-agent config firewall set \
  --pattern "https://analytics.com/.*" \
  --method GET \
  --mode mock \
  --action exclude

Q: What is the purpose of the init script?

A: The init script runs custom initialization logic during service startup, such as database setup, file preparation, or environment checks.

Example:

# .stoobly/services/my-service/mock/init
#!/bin/bash

# Wait for dependencies
sleep 2

# Setup test data
echo "Initializing test data..."
curl -X POST http://my-service/setup

# Verify service health
curl http://my-service/health

Q: What is the docker-compose.yml file used for?

A: The docker-compose.yml defines the service container, its dependencies, networks, environment variables, and how it connects to other services.

Example:

# .stoobly/services/my-service/mock/docker-compose.yml
services:
  my-service.proxy:
    depends_on:
      my-service.configure:
        condition: service_completed_successfully
    environment:
      - STOOBLY_HOSTNAME=my-service.local
      - STOOBLY_UPSTREAM_HOSTNAME=api.production.com
    networks:
      app.ingress: {}
    profiles:
      - ${WORKFLOW_NAME}

Q: What is the fixtures.yml file used for?

A: The fixtures.yml file contains mock response data for the service, used during mock and test workflows.

Example:

# .stoobly/services/my-service/mock/fixtures.yml
- GET:
  /users/d+?:
    headers: {}
    path: <RELATIVE-PATH-TO-TO-RESPONSE-FILE>
    status_code: 200
- POST:
  /users:
    headers: {}
    path: <RELATIVE-PATH-TO-TO-RESPONSE-FILE>
    status_code: 200

Q: What is the lifecycle_hooks.py file used for?

A: The lifecycle_hooks.py file contains Python functions that modify requests/responses during interception.

Example:

# .stoobly/services/my-service/mock/lifecycle_hooks.py

def before_request(context):
    # Modify request before sending
    context.request.headers['X-Custom-Header'] = 'test-value'
    return context

def after_response(context):
    # Modify response before returning
    if context.response.status_code == 404:
        context.response.status_code = 200
    return context

Q: What is the public/ directory used for?

A: The public/ directory serves static files for mocking, useful for serving images, CSS, JavaScript, or other assets.

Example:

# .stoobly/services/my-service/mock/public/
mkdir -p .stoobly/services/my-service/mock/public
echo '{"data": "test"}' > .stoobly/services/my-service/mock/public/test.json

# Access via: http://my-service.local/test.json

---

Workflow-Specific Configurations

Q: How are workflows organized?

A: Each service has separate directories for mock, record, and test workflows, allowing different configurations per workflow.

Example:

my-service/
├── mock/                       # Mock workflow config
│   ├── fixtures.yml           # Mock responses
│   └── docker-compose.yml     # Mock-specific settings
├── record/                     # Record workflow config
│   └── docker-compose.yml     # Record-specific settings
└── test/                       # Test workflow config
    ├── fixtures.yml           # Test fixtures
    └── docker-compose.yml     # Test-specific settings

Q: Can I have different configurations for different workflows?

A: Yes, each workflow directory has its own configure script, docker-compose.yml, and fixtures, allowing complete customization per workflow.

Example:

# Mock workflow - Use local fixtures
# .stoobly/services/my-service/mock/configure
stoobly-agent config rewrite set --mode mock --hostname localhost

# Record workflow - Use production
# .stoobly/services/my-service/record/configure
stoobly-agent config rewrite set --mode record --hostname api.production.com

# Test workflow - Use staging
# .stoobly/services/my-service/test/configure
stoobly-agent config rewrite set --mode test --hostname api.staging.com

---

Custom Workflows

Q: How do I create a custom workflow?

A: Use scaffold workflow create to create a new workflow based on a template (mock, record, or test).

Example:

# Create custom 'ci' workflow for a service
stoobly-agent scaffold workflow create ci \
  --template mock \
  --service my-service \
  --app-dir-path ./my-app

# Creates:
# .stoobly/services/my-service/ci/

Q: What gets created for a custom workflow?

A: A custom workflow gets the same structure as standard workflows: configure, docker-compose.yml, init, fixtures.yml, lifecycle_hooks.py, and public/.

Example:

.stoobly/services/my-service/ci/
├── configure
├── docker-compose.yml
├── fixtures.yml
├── init
├── lifecycle_hooks.py
└── public/

---

Temporary Runtime Files

Q: What is the .stoobly/tmp/ directory?

A: The tmp/ directory contains runtime files generated during workflow execution, including logs, run scripts, and Traefik configuration.

Example:

.stoobly/tmp/
├── mock/
│   ├── logs/
│   │   └── requests.json       # Request logs
│   ├── run.sh                  # Generated run script
│   └── traefik.yml             # Traefik config
├── record/
│   └── ...
└── test/
    └── ...

Q: What is the run.sh script?

A: The run.sh script is auto-generated by the scaffold workflow up command and contains the Docker Compose commands to start the workflow.

Example:

# Generated run.sh
#!/bin/bash
docker-compose -f .stoobly/services/build/mock/docker-compose.yml up
docker-compose -f .stoobly/services/gateway/mock/docker-compose.yml up -d
docker-compose -f .stoobly/services/my-service/mock/docker-compose.yml up -d
# ... more services

---

Practical Examples

Q: How do I add my application to the entrypoint service?

A: Edit the entrypoint docker-compose.yml to include your application container.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.my-app:
    image: node:18
    working_dir: /app
    volumes:
      - ../../..:/app
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
      - NO_PROXY=localhost,127.0.0.1
    profiles:
      - ${WORKFLOW_NAME}
    command: npm test

Q: How do I run my test suite in the entrypoint?

A: Configure the entrypoint service with your test command and ensure it uses the proxy.

Example:

# .stoobly/services/entrypoint/test/docker-compose.yml
services:
  entrypoint.playwright-tests:
    image: mcr.microsoft.com/playwright:latest
    working_dir: /tests
    volumes:
      - ../../..:/tests
    depends_on:
      entrypoint.configure:
        condition: service_completed_successfully
    networks:
      app.ingress: {}
    environment:
      - HTTP_PROXY=http://gateway:80
      - HTTPS_PROXY=http://gateway:80
    profiles:
      - ${WORKFLOW_NAME}
    command: npx playwright test

Q: How do I add environment variables to my service?

A: Add environment variables in the docker-compose.yml file for your service.

Example:

# .stoobly/services/my-service/mock/docker-compose.yml
services:
  my-service.app:
    environment:
      - DATABASE_URL=postgres://localhost/testdb
      - API_KEY=test-key-123
      - NODE_ENV=test
      - DEBUG=true

Q: How do I mount volumes for my service?

A: Add volume mounts in the docker-compose.yml to share files between host and container.

Example:

# .stoobly/services/my-service/mock/docker-compose.yml
services:
  my-service.app:
    volumes:
      - ../../..:/app                    # Mount project root
      - ./fixtures.yml:/app/fixtures.yml # Mount fixtures
      - ./public:/app/public             # Mount public files

---

Troubleshooting

Q: How do I debug scaffold issues?

A: Check the generated files in .stoobly/services/ and .stoobly/tmp/, and use --dry-run to see what commands would execute.

Example:

# Dry run to see commands
stoobly-agent scaffold workflow up mock --dry-run --app-dir-path ./my-app

# Check generated run script
cat .stoobly/tmp/mock/run.sh

# Check logs
cat .stoobly/tmp/mock/logs/requests.json

Q: Where can I find workflow logs?

A: Workflow logs are stored in .stoobly/tmp/<workflow>/logs/.

Example:

# View request logs
cat .stoobly/tmp/mock/logs/requests.json

# View Docker logs
docker-compose -f .stoobly/services/my-service/mock/docker-compose.yml logs

Q: How do I verify my service configuration?

A: Check the service's docker-compose.yml and run the configure script manually.

Example:

# View service config
cat .stoobly/services/my-service/mock/docker-compose.yml

# Run configure script
bash .stoobly/services/my-service/mock/configure

# Verify Stoobly config
stoobly-agent config dump

---

Quick Reference

Q: What are the key directories in a scaffold?

A: Here's a quick reference of important directories:

Example:

.stoobly/services/
├── Makefile                    # Main workflow commands
├── build/                      # Docker: Image building (Docker only)
├── entrypoint/                 # Docker: Your app runs here (Docker only)
├── gateway/                    # Docker: Traefik proxy (Docker only)
├── stoobly-ui/                 # Stoobly web UI (Both)
├── your-service/               # Your services (Both)
│   ├── mock/                   # Mock workflow
│   ├── record/                 # Record workflow
│   └── test/                   # Test workflow
└── .stoobly/tmp/               # Runtime files (Both)
    ├── mock/
    ├── record/
    └── test/

Q: What files can I customize?

A: You can customize configure scripts, docker-compose.yml, fixtures.yml, lifecycle_hooks.py, and add files to public/.

Example:

# Customizable files per service/workflow:
my-service/mock/
├── configure               # ✓ Customize Stoobly config
├── docker-compose.yml      # ✓ Add your containers
├── fixtures.yml            # ✓ Add mock responses
├── init                    # ✓ Add initialization logic
├── lifecycle_hooks.py      # ✓ Add request/response hooks
└── public/                 # ✓ Add static files