bifrost/tests/e2e/README.md

Bifrost E2E Tests

End-to-end tests for the Bifrost UI using Playwright.

Setup

# Install dependencies
npm install

# Install Playwright browsers
npx playwright install

Running Tests

# Run all E2E tests
make run-e2e

# Run specific feature tests
make run-e2e FLOW=providers
make run-e2e FLOW=virtual-keys
make run-e2e FLOW=dashboard
make run-e2e FLOW=logs
make run-e2e FLOW=mcp-logs
make run-e2e FLOW=mcp-registry
make run-e2e FLOW=routing-rules
make run-e2e FLOW=observability
make run-e2e FLOW=config
make run-e2e FLOW=plugins

# Run tests in headed mode (visible browser)
make run-e2e-headed

# Run tests with Playwright UI
make run-e2e-ui

# Run specific feature tests via npm
npm run test:providers
npm run test:virtual-keys
npm run test:dashboard
npm run test:logs
npm run test:mcp-logs
npm run test:mcp-registry
npm run test:routing-rules
npm run test:observability
npm run test:config
npm run test:plugins

# View test report
npm run report

Parallel flows on CI

The GitHub Actions workflow E2E Tests (.github/workflows/e2e-tests.yml) runs each flow in a separate job in parallel, since flows are independent. It triggers on push/PR when ui/, tests/e2e/, or the workflow file change. You can also run it manually (Actions → E2E Tests → Run workflow) and optionally pass a comma-separated list of flows (e.g. providers,config,plugins) to run only those.

Folder Structure

tests/e2e/
├── playwright.config.ts           # Playwright configuration
├── core/                          # Shared utilities & fixtures
│   ├── fixtures/                 # Custom test fixtures
│   ├── pages/                    # Base page objects
│   ├── actions/                  # Reusable actions
│   └── utils/                    # Utilities and helpers
└── features/                     # Feature-specific tests
    ├── providers/                # Provider tests
    ├── virtual-keys/             # Virtual key tests
    ├── dashboard/                # Dashboard tests
    ├── logs/                     # LLM logs tests
    ├── mcp-logs/                 # MCP logs tests
    ├── mcp-registry/             # MCP registry tests
    ├── routing-rules/            # Routing rules tests
    ├── plugins/                  # Plugins tests
    ├── observability/            # Observability connectors tests
    └── config/                   # Config settings tests

Writing Tests

Using Page Objects

import { test, expect } from '../../core/fixtures/base.fixture'

test('should create provider', async ({ providersPage }) => {
  await providersPage.goto()
  await providersPage.selectProvider('openai')
  // ...
})

Test Data

Use factory functions from the *.data.ts files for generating test data:

import { createProviderKeyData } from './providers.data'

const keyData = createProviderKeyData({ name: 'My Key' })

Configuration

Environment variables:

• BASE_URL - Base URL of the application (default: http://localhost:3000)
• CI - Set to true in CI environments

Debugging

# Run with Playwright Inspector
npm run test:debug

# Generate code with Codegen
npm run codegen

Best Practices

Wait Strategies

Use semantic waits instead of hardcoded timeouts:

// ✅ Good: Semantic waits
await page.waitForLoadState('networkidle')
await element.waitFor({ state: 'visible' })
await expect(element).toBeVisible({ timeout: 5000 })

// ❌ Bad: Hardcoded timeouts (flaky and slow)
await page.waitForTimeout(2000)

Selectors

Use data-testid attributes for robust selectors:

// ✅ Good: Test IDs are resilient to UI changes
page.locator('[data-testid="chart-log-volume"]')
page.getByTestId('create-btn')

// ❌ Bad: Brittle chained parent selectors
page.locator('text=Volume').locator('..').locator('..')

Resource Cleanup

Always clean up resources created during tests:

// ✅ Good: Clean up after assertions
test('should create item', async ({ page }) => {
  await page.createItem(data)
  expect(await page.itemExists(data.name)).toBe(true)
  // Cleanup
  await page.deleteItem(data.name)
})

Deterministic Assertions

Avoid conditional logic that always passes:

// ❌ Bad: Always passes (count >= 0 is always true)
const count = await page.getCount()
expect(count >= 0).toBe(true)

// ✅ Good: Deterministic assertion
const count = await page.getCount()
if (count === 0) {
  expect(emptyState).toBeVisible()
} else {
  expect(count).toBeGreaterThan(0)
  expect(emptyState).not.toBeVisible()
}

Anti-Patterns to Avoid

1. waitForTimeout() - Always use semantic waits instead
2. { force: true } - Fix underlying visibility issues instead
3. Chained parent locators (.locator('..')) - Use data-testid attributes
4. Conditional assertions that always pass - Write deterministic tests
5. Static test data names - Use timestamps for uniqueness
6. Missing cleanup - Delete created resources to prevent pollution

Troubleshooting

Tests Failing Intermittently

1. Replace waitForTimeout() with proper semantic waits
2. Ensure toasts are dismissed: await page.dismissToasts()
3. Add waitForPageLoad() after navigation
4. Wait for sheets/modals to complete animation: await page.waitForSheetAnimation()

Tests Pass Individually but Fail Together

1. Add cleanup for created resources
2. Use unique names with Date.now() timestamps
3. Check for leftover state from previous tests

Element Not Clickable

1. Ensure element is visible: await element.waitFor({ state: 'visible' })
2. Scroll element into view: await element.scrollIntoViewIfNeeded()
3. Dismiss overlaying toasts: await page.dismissToasts()
4. Don't use { force: true } - fix the root cause