E2E Testing
Stoobly Scaffold E2E Testing - Questions & Answers
Stoobly scaffold provides first-class support for E2E testing frameworks like Playwright and Cypress, enabling you to record, mock, and validate API interactions during your end-to-end tests.
📚 Related Documentation:
⚠️ Important: Make commands (e.g., make -f .stoobly/services/Makefile test) are Docker-specific only. Local runtime uses CLI commands directly (e.g., stoobly-agent scaffold workflow up test).
Plugin Support
Q: What E2E testing frameworks does Stoobly support?
A: Stoobly supports Playwright and Cypress through the --plugin option when creating an app.
Example:
# Create app with Playwright support
stoobly-agent scaffold app create my-app --plugin playwright
# Create app with Cypress support
stoobly-agent scaffold app create my-app --plugin cypress
# Create app with both plugins
stoobly-agent scaffold app create my-app --plugin playwright --plugin cypressNote: The files created depend on whether you use --runtime docker or --runtime local. For Docker-specific file details, see docker.md. For local runtime setup, see local.md.
Frontend Service Setup
Q: How do I set up a frontend service to serve static assets for E2E tests?
A: Frontend services typically only need the test workflow since they serve static assets from fixtures rather than recording or mocking requests. Create the service with --workflow test and modify the test/init script to copy your built application to the public fixtures folder.
Example:
Modify the init script to copy built assets:
Why only test workflow?
Frontend services serve static assets, not dynamic API responses
No need to record requests (no API logic to capture)
No need to mock responses (serving files, not making requests)
Test workflow is needed so E2E tests can request the application under test
Configuration Files
Q: What files should I modify to add my Playwright tests?
A: The files you modify depend on your runtime:
Docker runtime: Modify entrypoint docker-compose.yml, configure script, and Playwright config. See docker.md for details.
Local runtime: Configure Playwright to use Stoobly's proxy via environment variables or config. See local.md for details.
Q: What files should I modify to add my Cypress tests?
A: The files you modify depend on your runtime:
Docker runtime: Modify entrypoint docker-compose.yml and Cypress config. See docker.md for details.
Local runtime: Configure Cypress to use Stoobly's proxy via environment variables or config. See local.md for details.
Q: How do I add custom npm packages for my tests?
A: The approach depends on your runtime:
Docker runtime: Use custom Docker images with
PLAYWRIGHT_IMAGEorCYPRESS_IMAGEenvironment variables. See docker.md for details.Local runtime: Install packages directly in your project using npm/yarn. No special configuration needed.
Q: How do I pass environment variables to my tests?
A: The approach depends on your runtime:
Docker runtime: Add environment variables in docker-compose.yml. See docker.md for details.
Local runtime: Set environment variables in your shell or use a
.envfile. See local.md for details.
Running E2E Tests
Q: How do I record E2E test traffic?
A: Use the record workflow to capture all HTTP requests made during your E2E tests.
Q: How do I run E2E tests with mocked responses?
A: Use the mock workflow to run tests against recorded responses without hitting real APIs.
Q: How do I run E2E tests with response validation?
A: Use the test workflow to validate that responses match expected results.
Q: How do I run specific test files or suites?
A: The approach depends on your runtime:
Docker runtime: Modify the command in docker-compose.yml. See docker.md for details.
Local runtime: Use your test framework's command-line options. See local.md for details.
Debugging
Q: How do I view test output and logs?
A: The approach depends on your runtime:
Docker runtime: Use Makefile logs commands. See docker.md for details.
Local runtime: View logs from the workflow directory or Stoobly agent. See local.md for details.
Q: How do I debug failing tests?
A: The approach depends on your runtime:
Docker runtime: Enable headed mode and increase logging verbosity in docker-compose.yml. See docker.md for details.
Local runtime: Enable debug mode in your test framework. See local.md for details.
Q: How do I save test artifacts (screenshots, videos)?
A: The approach depends on your runtime:
Docker runtime: Mount an artifacts directory and configure your test framework. See docker.md for details.
Local runtime: Configure your test framework to save artifacts to a local directory. See local.md for details.
Advanced Configuration
Q: How do I mount test fixtures and data?
A: The approach depends on your runtime:
Docker runtime: Add volume mounts to share fixtures between host and container. See docker.md for details.
Local runtime: Reference fixtures and test data using relative paths from your project root. See local.md for details.
Q: How do I handle dynamic data in E2E tests?
A: Use Stoobly's rewrite rules to normalize dynamic values in the configure script. This applies to both Docker and local runtimes.
Example:
Best Practices
Q: Should I record once and mock for all subsequent runs?
A: Yes, this is the recommended approach for fast, reliable E2E tests that don't depend on external APIs.
Docker runtime: See docker.md for the complete workflow example.
Local runtime: See local.md for the complete workflow example.
Q: How do I organize E2E tests by feature?
A: Create separate services or custom workflows for different test suites.
CI/CD Integration
Q: Which runtime should I use in CI/CD for E2E testing?
A: Docker is recommended for CI/CD as it provides consistent, isolated environments. Local runtime can also be used if your CI environment has Python and stoobly-agent installed.
Q: How do I test with different runtimes in CI/CD?
A: Create separate CI jobs, each using the desired runtime.
Example:
Complete Examples
Q: What's a complete example of setting up Playwright E2E tests?
A: Complete examples are available for both runtimes:
Docker runtime: See docker.md for a complete Docker-based example.
Local runtime: See local.md for local runtime setup guidance.
Quick Reference
Key files to modify for E2E testing:
Docker runtime: See docker.md for file structure and customization details.
Local runtime: See local.md for file structure and customization details.
Common commands:
Docker runtime (Makefile commands): See docker.md for complete command reference.
Local runtime (CLI commands): See local.md for complete command reference.
For more details:
Last updated
Was this helpful?