Unverified 提交 b2bf3bed authored 作者: wwwillchen-bot's avatar wwwillchen-bot 提交者: GitHub

refactor: modularize e2e test helpers into separate modules (#2556)

## Summary - Split the monolithic `test_helper.ts` (~1700 lines) into focused modules for better maintainability - Created `constants.ts` for timeout constants - Created `fixtures.ts` for Playwright fixtures and test setup - Created `utils/` directory for utility functions (normalization, dump-prettifier) - Created `page-objects/` directory with component and dialog page objects ## Test plan - [x] All existing unit tests pass (784 tests) - [x] Lint and type checks pass - [ ] E2E tests should work with the refactored helpers (imports are re-exported from test_helper.ts for backward compatibility) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/dyad-sh/dyad/pull/2556" target="_blank"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end --> <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Refactored e2e test helpers into modular page objects, fixtures, and utilities for easier maintenance and more deterministic snapshots. Added loop guards to prevent hangs in toast dismissal and the Context Files Picker; existing tests keep working via re-exports. - **Refactors** - Split test_helper.ts into constants.ts, fixtures.ts, utils/, and page-objects/ (components + dialogs); added a main PageObject that composes component page objects. - Introduced normalization and dump-prettifier utilities for stable snapshots. - Centralized CI-aware timeouts in constants.ts; added optional debug logging flag. - Kept test_helper.ts as a thin re-export for backward compatibility. - Updated TypeScript strict-mode docs with ES2020 target limitations on replaceAll. - **Bug Fixes** - Fixed importApp path resolution to use __dirname three levels up. - Switched git config calls to execFileSync to avoid command injection. - Corrected toast dismissal loop to click the first toast repeatedly until none remain; added maxAttempts=20 guard (and in Context Files Picker) to prevent hangs. - Derived the app executable name from appInfo instead of hardcoding dyad.exe. <sup>Written for commit 26329e6b1866fd31e1cba91e0b641fc04df30e8a. Summary will update on new commits.</sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: 's avatarWill Chen <willchen90@gmail.com> Co-authored-by: 's avatarClaude Opus 4.5 <noreply@anthropic.com> Co-authored-by: 's avatarclaude[bot] <41898282+claude[bot]@users.noreply.github.com>
上级 99b2f74b
/**
* Timeout constants for e2e tests.
* Values are adjusted for CI vs local environments.
*/
export const Timeout = {
// Things generally take longer on CI, so we make them longer.
EXTRA_LONG: process.env.CI ? 120_000 : 60_000,
LONG: process.env.CI ? 60_000 : 30_000,
MEDIUM: process.env.CI ? 30_000 : 15_000,
SHORT: process.env.CI ? 5_000 : 2_000,
};
export const showDebugLogs = process.env.DEBUG_LOGS === "true";
/**
* Playwright test fixtures for e2e tests.
* Provides Electron app launching and PageObject initialization.
*/
import { test as base } from "@playwright/test";
import * as eph from "electron-playwright-helpers";
import { ElectronApplication, _electron as electron } from "playwright";
import os from "os";
import path from "path";
import { execSync } from "child_process";
import { showDebugLogs } from "./constants";
import { PageObject } from "./page-objects";
export interface ElectronConfig {
preLaunchHook?: ({ userDataDir }: { userDataDir: string }) => Promise<void>;
showSetupScreen?: boolean;
}
// From https://github.com/microsoft/playwright/issues/8208#issuecomment-1435475930
//
// Note how we mark the fixture as { auto: true }.
// This way it is always instantiated, even if the test does not use it explicitly.
export const test = base.extend<{
electronConfig: ElectronConfig;
attachScreenshotsToReport: void;
electronApp: ElectronApplication;
po: PageObject;
}>({
electronConfig: [
async ({}, use) => {
// Default configuration - tests can override this fixture
await use({});
},
{ auto: true },
],
po: [
async ({ electronApp }, use) => {
const page = await electronApp.firstWindow();
const po = new PageObject(electronApp, page, {
userDataDir: (electronApp as any).$dyadUserDataDir,
});
await use(po);
},
{ auto: true },
],
attachScreenshotsToReport: [
async ({ electronApp }, use, testInfo) => {
await use();
// After the test we can check whether the test passed or failed.
if (testInfo.status !== testInfo.expectedStatus) {
const page = await electronApp.firstWindow();
try {
const screenshot = await page.screenshot({ timeout: 5_000 });
await testInfo.attach("screenshot", {
body: screenshot,
contentType: "image/png",
});
} catch (error) {
console.error("Error taking screenshot on failure", error);
}
}
},
{ auto: true },
],
electronApp: [
async ({ electronConfig }, use) => {
// find the latest build in the out directory
const latestBuild = eph.findLatestBuild();
// parse the directory and find paths and other info
const appInfo = eph.parseElectronApp(latestBuild);
process.env.OLLAMA_HOST = "http://localhost:3500/ollama";
process.env.LM_STUDIO_BASE_URL_FOR_TESTING =
"http://localhost:3500/lmstudio";
process.env.DYAD_ENGINE_URL = "http://localhost:3500/engine/v1";
process.env.DYAD_GATEWAY_URL = "http://localhost:3500/gateway/v1";
process.env.E2E_TEST_BUILD = "true";
if (!electronConfig.showSetupScreen) {
// This is just a hack to avoid the AI setup screen.
process.env.OPENAI_API_KEY = "sk-test";
}
const baseTmpDir = os.tmpdir();
const userDataDir = path.join(baseTmpDir, `dyad-e2e-tests-${Date.now()}`);
if (electronConfig.preLaunchHook) {
await electronConfig.preLaunchHook({ userDataDir });
}
const electronApp = await electron.launch({
args: [
appInfo.main,
"--enable-logging",
`--user-data-dir=${userDataDir}`,
],
executablePath: appInfo.executable,
// Strong suspicion this is causing issues on Windows with tests hanging due to error:
// ffmpeg failed to write: Error [ERR_STREAM_WRITE_AFTER_END]: write after end
// recordVideo: {
// dir: "test-results",
// },
});
(electronApp as any).$dyadUserDataDir = userDataDir;
console.log("electronApp launched!");
if (showDebugLogs) {
// Listen to main process output immediately
electronApp.process().stdout?.on("data", (data) => {
console.log(`MAIN_PROCESS_STDOUT: ${data.toString()}`);
});
electronApp.process().stderr?.on("data", (data) => {
console.error(`MAIN_PROCESS_STDERR: ${data.toString()}`);
});
}
electronApp.on("close", () => {
console.log(`Electron app closed listener:`);
});
electronApp.on("window", async (page) => {
const filename = page.url()?.split("/").pop();
console.log(`Window opened: ${filename}`);
// capture errors
page.on("pageerror", (error) => {
console.error(error);
});
// capture console messages
page.on("console", (msg) => {
console.log(msg.text());
});
});
await use(electronApp);
// Why are we doing a force kill on Windows?
//
// Otherwise, Playwright will just hang on the test cleanup
// because the electron app does NOT ever fully quit due to
// Windows' strict resource locking (e.g. file locking).
if (os.platform() === "win32") {
try {
const executableName = path.basename(appInfo.executable);
console.log(`[cleanup:start] Killing ${executableName}`);
console.time("taskkill");
execSync(`taskkill /f /t /im ${executableName}`);
console.timeEnd("taskkill");
console.log(`[cleanup:end] Killed ${executableName}`);
} catch (error) {
console.warn(
"Failed to kill dyad.exe: (continuing with test cleanup)",
error,
);
}
} else {
await electronApp.close();
}
},
{ auto: true },
],
});
/**
* Creates a test with custom Electron configuration.
*/
export function testWithConfig(config: ElectronConfig) {
return test.extend({
electronConfig: async ({}, use) => {
await use(config);
},
});
}
/**
* Creates a test with custom Electron configuration, but skips on Windows.
*/
export function testWithConfigSkipIfWindows(config: ElectronConfig) {
if (os.platform() === "win32") {
return test.skip;
}
return test.extend({
electronConfig: async ({}, use) => {
await use(config);
},
});
}
/**
* Wrapper that skips tests on Windows platform.
*/
export const testSkipIfWindows = os.platform() === "win32" ? test.skip : test;
/**
* Main PageObject class that composes all component page objects.
* This provides a single entry point for tests while maintaining
* backward compatibility with the existing API.
*/
import { Page, expect } from "@playwright/test";
import { ElectronApplication } from "playwright";
import fs from "fs";
import { generateAppFilesSnapshotData } from "../generateAppFilesSnapshotData";
import {
normalizeItemReferences,
normalizeToolCallIds,
normalizeVersionedFiles,
normalizePath,
prettifyDump,
} from "../utils";
// Import component page objects
import { GitHubConnector } from "./components/GitHubConnector";
import { ChatActions } from "./components/ChatActions";
import { PreviewPanel } from "./components/PreviewPanel";
import { CodeEditor } from "./components/CodeEditor";
import { SecurityReview } from "./components/SecurityReview";
import { ToastNotifications } from "./components/ToastNotifications";
import { AgentConsent } from "./components/AgentConsent";
import { Navigation } from "./components/Navigation";
import { ModelPicker } from "./components/ModelPicker";
import { Settings } from "./components/Settings";
import { AppManagement } from "./components/AppManagement";
import { PromptLibrary } from "./components/PromptLibrary";
// Import dialog page objects
import { ContextFilesPickerDialog } from "./dialogs/ContextFilesPickerDialog";
import { ProModesDialog } from "./dialogs/ProModesDialog";
export class PageObject {
public userDataDir: string;
// Component page objects (exposed for direct access if needed)
public githubConnector: GitHubConnector;
private chatActions: ChatActions;
private previewPanel: PreviewPanel;
private codeEditor: CodeEditor;
private securityReview: SecurityReview;
private toastNotifications: ToastNotifications;
private agentConsent: AgentConsent;
private navigation: Navigation;
private modelPicker: ModelPicker;
private settings: Settings;
private appManagement: AppManagement;
private promptLibrary: PromptLibrary;
constructor(
public electronApp: ElectronApplication,
public page: Page,
{ userDataDir }: { userDataDir: string },
) {
this.userDataDir = userDataDir;
// Initialize component page objects
this.githubConnector = new GitHubConnector(this.page);
this.chatActions = new ChatActions(this.page);
this.previewPanel = new PreviewPanel(this.page);
this.codeEditor = new CodeEditor(this.page);
this.securityReview = new SecurityReview(this.page);
this.toastNotifications = new ToastNotifications(this.page);
this.agentConsent = new AgentConsent(this.page);
this.navigation = new Navigation(this.page);
this.modelPicker = new ModelPicker(this.page);
this.settings = new Settings(this.page, userDataDir);
this.appManagement = new AppManagement(this.page, electronApp, userDataDir);
this.promptLibrary = new PromptLibrary(this.page);
}
// ================================
// Setup Methods
// ================================
private async baseSetup() {
await this.githubConnector.clearPushEvents();
}
async setUp({
autoApprove = false,
disableNativeGit = false,
enableAutoFixProblems = false,
enableBasicAgent = false,
}: {
autoApprove?: boolean;
disableNativeGit?: boolean;
enableAutoFixProblems?: boolean;
enableBasicAgent?: boolean;
} = {}) {
await this.baseSetup();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
if (disableNativeGit) {
await this.toggleNativeGit();
}
if (enableAutoFixProblems) {
await this.toggleAutoFixProblems();
}
await this.setUpTestProvider();
await this.setUpTestModel();
await this.goToAppsTab();
if (!enableBasicAgent) {
await this.selectChatMode("build");
}
await this.selectTestModel();
}
async setUpDyadPro({
autoApprove = false,
localAgent = false,
localAgentUseAutoModel = false,
}: {
autoApprove?: boolean;
localAgent?: boolean;
localAgentUseAutoModel?: boolean;
} = {}) {
await this.baseSetup();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
await this.setUpDyadProvider();
await this.goToAppsTab();
if (!localAgent) {
await this.selectChatMode("build");
}
// Select a non-openAI model for local agent mode,
// since openAI models go to the responses API.
if (localAgent && !localAgentUseAutoModel) {
await this.selectModel({
provider: "Anthropic",
model: "Claude Opus 4.5",
});
}
}
async setUpAzure({ autoApprove = false }: { autoApprove?: boolean } = {}) {
await this.githubConnector.clearPushEvents();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
// Azure should already be configured via environment variables
// so we don't need additional setup steps like setUpDyadProvider
await this.goToAppsTab();
}
// ================================
// Chat Actions (delegated)
// ================================
getHomeChatInputContainer() {
return this.chatActions.getHomeChatInputContainer();
}
getChatInputContainer() {
return this.chatActions.getChatInputContainer();
}
getChatInput() {
return this.chatActions.getChatInput();
}
async clearChatInput() {
return this.chatActions.clearChatInput();
}
async openChatHistoryMenu() {
return this.chatActions.openChatHistoryMenu();
}
clickNewChat(options?: { index?: number }) {
return this.chatActions.clickNewChat(options);
}
async waitForChatCompletion() {
return this.chatActions.waitForChatCompletion();
}
async clickRetry() {
return this.chatActions.clickRetry();
}
async clickUndo() {
return this.chatActions.clickUndo();
}
async sendPrompt(
prompt: string,
options?: { skipWaitForCompletion?: boolean },
) {
return this.chatActions.sendPrompt(prompt, options);
}
async selectChatMode(
mode: "build" | "ask" | "agent" | "local-agent" | "basic-agent" | "plan",
) {
return this.chatActions.selectChatMode(mode);
}
async selectLocalAgentMode() {
return this.chatActions.selectLocalAgentMode();
}
async clickChatActivityButton() {
return this.chatActions.clickChatActivityButton();
}
async snapshotChatActivityList() {
return this.chatActions.snapshotChatActivityList();
}
async snapshotChatInputContainer() {
return this.chatActions.snapshotChatInputContainer();
}
// ================================
// Preview Panel (delegated)
// ================================
async selectPreviewMode(
mode:
| "code"
| "problems"
| "preview"
| "configure"
| "security"
| "publish",
) {
return this.previewPanel.selectPreviewMode(mode);
}
async clickRecheckProblems() {
return this.previewPanel.clickRecheckProblems();
}
async clickFixAllProblems() {
await this.previewPanel.clickFixAllProblems();
await this.waitForChatCompletion();
}
async snapshotProblemsPane() {
return this.previewPanel.snapshotProblemsPane();
}
async clickRebuild() {
return this.previewPanel.clickRebuild();
}
async clickTogglePreviewPanel() {
return this.previewPanel.clickTogglePreviewPanel();
}
async clickPreviewPickElement() {
return this.previewPanel.clickPreviewPickElement();
}
async clickDeselectComponent(options?: { index?: number }) {
return this.previewPanel.clickDeselectComponent(options);
}
async clickPreviewMoreOptions() {
return this.previewPanel.clickPreviewMoreOptions();
}
async clickPreviewRefresh() {
return this.previewPanel.clickPreviewRefresh();
}
async clickPreviewNavigateBack() {
return this.previewPanel.clickPreviewNavigateBack();
}
async clickPreviewNavigateForward() {
return this.previewPanel.clickPreviewNavigateForward();
}
async clickPreviewOpenBrowser() {
return this.previewPanel.clickPreviewOpenBrowser();
}
async clickPreviewAnnotatorButton() {
return this.previewPanel.clickPreviewAnnotatorButton();
}
async waitForAnnotatorMode() {
return this.previewPanel.waitForAnnotatorMode();
}
async clickAnnotatorSubmit() {
return this.previewPanel.clickAnnotatorSubmit();
}
locateLoadingAppPreview() {
return this.previewPanel.locateLoadingAppPreview();
}
locateStartingAppPreview() {
return this.previewPanel.locateStartingAppPreview();
}
getPreviewIframeElement() {
return this.previewPanel.getPreviewIframeElement();
}
expectPreviewIframeIsVisible() {
return this.previewPanel.expectPreviewIframeIsVisible();
}
async clickFixErrorWithAI() {
return this.previewPanel.clickFixErrorWithAI();
}
async clickCopyErrorMessage() {
return this.previewPanel.clickCopyErrorMessage();
}
async clickFixAllErrors() {
return this.previewPanel.clickFixAllErrors();
}
async snapshotPreviewErrorBanner() {
return this.previewPanel.snapshotPreviewErrorBanner();
}
locatePreviewErrorBanner() {
return this.previewPanel.locatePreviewErrorBanner();
}
getSelectedComponentsDisplay() {
return this.previewPanel.getSelectedComponentsDisplay();
}
async snapshotSelectedComponentsDisplay() {
return this.previewPanel.snapshotSelectedComponentsDisplay();
}
async snapshotPreview(options?: { name?: string }) {
return this.previewPanel.snapshotPreview(options);
}
// ================================
// Code Editor (delegated)
// ================================
async clickEditButton() {
return this.codeEditor.clickEditButton();
}
async editFileContent(content: string) {
return this.codeEditor.editFileContent(content);
}
async saveFile() {
return this.codeEditor.saveFile();
}
async cancelEdit() {
return this.codeEditor.cancelEdit();
}
// ================================
// Security Review (delegated)
// ================================
async clickRunSecurityReview() {
return this.securityReview.clickRunSecurityReview();
}
async snapshotSecurityFindingsTable() {
return this.securityReview.snapshotSecurityFindingsTable();
}
// ================================
// Toast Notifications (delegated)
// ================================
async expectNoToast() {
return this.toastNotifications.expectNoToast();
}
async waitForToast(
type?: "success" | "error" | "warning" | "info",
timeout?: number,
) {
return this.toastNotifications.waitForToast(type, timeout);
}
async waitForToastWithText(text: string, timeout?: number) {
return this.toastNotifications.waitForToastWithText(text, timeout);
}
async assertToastVisible(type?: "success" | "error" | "warning" | "info") {
return this.toastNotifications.assertToastVisible(type);
}
async assertToastWithText(text: string) {
return this.toastNotifications.assertToastWithText(text);
}
async dismissAllToasts() {
return this.toastNotifications.dismissAllToasts();
}
// ================================
// Agent Consent (delegated)
// ================================
getAgentConsentBanner() {
return this.agentConsent.getAgentConsentBanner();
}
async waitForAgentConsentBanner(timeout?: number) {
return this.agentConsent.waitForAgentConsentBanner(timeout);
}
async clickAgentConsentAlwaysAllow() {
return this.agentConsent.clickAgentConsentAlwaysAllow();
}
async clickAgentConsentAllowOnce() {
return this.agentConsent.clickAgentConsentAllowOnce();
}
async clickAgentConsentDecline() {
return this.agentConsent.clickAgentConsentDecline();
}
// ================================
// Navigation (delegated)
// ================================
async goToSettingsTab() {
return this.navigation.goToSettingsTab();
}
async goToLibraryTab() {
return this.navigation.goToLibraryTab();
}
async goToAppsTab() {
return this.navigation.goToAppsTab();
}
async goToChatTab() {
return this.navigation.goToChatTab();
}
async goToHubTab() {
return this.navigation.goToHubTab();
}
async clickBackButton() {
return this.navigation.clickBackButton();
}
async selectTemplate(templateName: string) {
return this.navigation.selectTemplate(templateName);
}
async goToHubAndSelectTemplate(templateName: "Next.js Template") {
return this.navigation.goToHubAndSelectTemplate(templateName);
}
// ================================
// Model Picker (delegated)
// ================================
async selectModel(options: { provider: string; model: string }) {
return this.modelPicker.selectModel(options);
}
async selectTestModel() {
return this.modelPicker.selectTestModel();
}
async selectTestOllamaModel() {
return this.modelPicker.selectTestOllamaModel();
}
async selectTestLMStudioModel() {
return this.modelPicker.selectTestLMStudioModel();
}
async selectTestAzureModel() {
return this.modelPicker.selectTestAzureModel();
}
// ================================
// Settings (delegated)
// ================================
async toggleAutoApprove() {
return this.settings.toggleAutoApprove();
}
async toggleLocalAgentMode() {
return this.settings.toggleLocalAgentMode();
}
async toggleNativeGit() {
return this.settings.toggleNativeGit();
}
async toggleAutoFixProblems() {
return this.settings.toggleAutoFixProblems();
}
async toggleAutoUpdate() {
return this.settings.toggleAutoUpdate();
}
async changeReleaseChannel(channel: "stable" | "beta") {
return this.settings.changeReleaseChannel(channel);
}
async clickTelemetryAccept() {
return this.settings.clickTelemetryAccept();
}
async clickTelemetryReject() {
return this.settings.clickTelemetryReject();
}
async clickTelemetryLater() {
return this.settings.clickTelemetryLater();
}
recordSettings() {
return this.settings.recordSettings();
}
snapshotSettingsDelta(beforeSettings: Record<string, unknown>) {
return this.settings.snapshotSettingsDelta(beforeSettings);
}
async setUpTestProvider() {
return this.settings.setUpTestProvider();
}
async setUpTestModel() {
return this.settings.setUpTestModel();
}
async addCustomTestModel(options: { name: string; contextWindow?: number }) {
return this.settings.addCustomTestModel(options);
}
async setUpTestProviderApiKey() {
return this.settings.setUpTestProviderApiKey();
}
async setUpDyadProvider() {
return this.settings.setUpDyadProvider();
}
// ================================
// App Management (delegated)
// ================================
getTitleBarAppNameButton() {
return this.appManagement.getTitleBarAppNameButton();
}
getAppListItem(options: { appName: string }) {
return this.appManagement.getAppListItem(options);
}
async isCurrentAppNameNone() {
return this.appManagement.isCurrentAppNameNone();
}
async getCurrentAppName() {
return this.appManagement.getCurrentAppName();
}
async getCurrentAppPath() {
return this.appManagement.getCurrentAppPath();
}
getAppPath(options: { appName: string }) {
return this.appManagement.getAppPath(options);
}
async clickAppListItem(options: { appName: string }) {
return this.appManagement.clickAppListItem(options);
}
async clickOpenInChatButton() {
return this.appManagement.clickOpenInChatButton();
}
locateAppUpgradeButton(options: { upgradeId: string }) {
return this.appManagement.locateAppUpgradeButton(options);
}
async clickAppUpgradeButton(options: { upgradeId: string }) {
return this.appManagement.clickAppUpgradeButton(options);
}
async expectAppUpgradeButtonIsNotVisible(options: { upgradeId: string }) {
return this.appManagement.expectAppUpgradeButtonIsNotVisible(options);
}
async expectNoAppUpgrades() {
return this.appManagement.expectNoAppUpgrades();
}
async clickAppDetailsRenameAppButton() {
return this.appManagement.clickAppDetailsRenameAppButton();
}
async clickAppDetailsMoreOptions() {
return this.appManagement.clickAppDetailsMoreOptions();
}
async clickAppDetailsCopyAppButton() {
return this.appManagement.clickAppDetailsCopyAppButton();
}
async clickConnectSupabaseButton() {
return this.appManagement.clickConnectSupabaseButton();
}
async importApp(appDir: string) {
return this.appManagement.importApp(appDir);
}
async configureGitUser(options?: {
email?: string;
name?: string;
disableGpgSign?: boolean;
}) {
return this.appManagement.configureGitUser(options);
}
async ensurePnpmInstall() {
return this.appManagement.ensurePnpmInstall();
}
// ================================
// Prompt Library (delegated)
// ================================
async createPrompt(options: {
title: string;
description?: string;
content: string;
}) {
return this.promptLibrary.createPrompt(options);
}
// ================================
// Dialog Openers
// ================================
async openContextFilesPicker() {
// Programmatically dismiss toasts using the sonner API by clicking any visible close buttons
const toastCloseButtons = this.page.locator(
"[data-sonner-toast] button[data-close-button]",
);
const maxAttempts = 20;
let attempts = 0;
while ((await toastCloseButtons.count()) > 0 && attempts < maxAttempts) {
await toastCloseButtons
.first()
.click()
.catch(() => {});
attempts++;
}
// If close buttons don't work, click outside to dismiss
if ((await this.page.locator("[data-sonner-toast]").count()) > 0) {
// Click somewhere safe to dismiss toasts
await this.page.mouse.click(10, 10);
await this.page.waitForTimeout(300);
}
// Open the auxiliary actions menu
await this.getChatInputContainer()
.getByTestId("auxiliary-actions-menu")
.click();
// Click on "Codebase context" to open the popover
await this.page.getByTestId("codebase-context-trigger").click();
// Wait for the popover content to be visible
await this.page
.getByTestId("manual-context-files-input")
.waitFor({ state: "visible" });
return new ContextFilesPickerDialog(this.page, async () => {
// Close the popover first
await this.page.keyboard.press("Escape");
// Wait a bit for the popover to close, then close the dropdown menu
await this.page
.getByTestId("manual-context-files-input")
.waitFor({ state: "hidden" });
await this.page.keyboard.press("Escape");
});
}
async openProModesDialog({
location = "chat-input-container",
}: {
location?: "chat-input-container" | "home-chat-input-container";
} = {}): Promise<ProModesDialog> {
const proButton = this.page
// Assumes you're on the chat page.
.getByTestId(location)
.getByRole("button", { name: "Pro", exact: true });
await proButton.click();
return new ProModesDialog(this.page, async () => {
await proButton.click();
});
}
// ================================
// Proposal Actions
// ================================
async approveProposal() {
await this.page.getByTestId("approve-proposal-button").click();
}
async rejectProposal() {
await this.page.getByTestId("reject-proposal-button").click();
}
async clickRestart() {
await this.page.getByRole("button", { name: "Restart" }).click();
}
// ================================
// Token Bar
// ================================
async toggleTokenBar() {
// Need to make sure it's NOT visible yet to avoid a race when we opened
// the auxiliary actions menu earlier.
await expect(this.page.getByTestId("token-bar-toggle")).not.toBeVisible();
await this.getChatInputContainer()
.getByTestId("auxiliary-actions-menu")
.click();
await this.page.getByTestId("token-bar-toggle").click();
}
// ================================
// Clipboard
// ================================
async getClipboardText(): Promise<string> {
return await this.page.evaluate(() => navigator.clipboard.readText());
}
// ================================
// Utility Methods
// ================================
async sleep(ms: number) {
await new Promise((resolve) => setTimeout(resolve, ms));
}
// ================================
// Snapshot Methods
// ================================
async snapshotDialog() {
await expect(this.page.getByRole("dialog")).toMatchAriaSnapshot();
}
async snapshotAppFiles({ name, files }: { name: string; files?: string[] }) {
const currentAppName = await this.getCurrentAppName();
if (!currentAppName) {
throw new Error("No app selected");
}
const normalizedAppName = currentAppName.toLowerCase().replace(/-/g, "");
const appPath = await this.getCurrentAppPath();
if (!appPath || !fs.existsSync(appPath)) {
throw new Error(`App path does not exist: ${appPath}`);
}
await expect(() => {
let filesData = generateAppFilesSnapshotData(appPath, appPath);
// Sort by relative path to ensure deterministic output
filesData.sort((a, b) => a.relativePath.localeCompare(b.relativePath));
if (files) {
filesData = filesData.filter((file) =>
files.some(
(f) => normalizePath(f) === normalizePath(file.relativePath),
),
);
}
const snapshotContent = filesData
.map(
(file) =>
`=== ${file.relativePath.replace(normalizedAppName, "[[normalizedAppName]]")} ===\n${file.content
.split(normalizedAppName)
.join("[[normalizedAppName]]")
.split(currentAppName)
.join("[[appName]]")}`,
)
.join("\n\n");
if (name) {
expect(snapshotContent).toMatchSnapshot(name + ".txt");
} else {
expect(snapshotContent).toMatchSnapshot();
}
}).toPass();
}
async snapshotMessages({
replaceDumpPath = false,
timeout,
}: { replaceDumpPath?: boolean; timeout?: number } = {}) {
// NOTE: once you have called this, you can NOT manipulate the UI anymore or React will break.
if (replaceDumpPath) {
await this.page.evaluate(() => {
const messagesList = document.querySelector(
"[data-testid=messages-list]",
);
if (!messagesList) {
throw new Error("Messages list not found");
}
// Scrub compaction backup paths embedded in message text
// e.g. .dyad/chats/1/compaction-2026-02-05T21-25-24-285Z.md
messagesList.innerHTML = messagesList.innerHTML.replace(
/\.dyad\/chats\/\d+\/compaction-[^\s<"]+\.md/g,
"[[compaction-backup-path]]",
);
messagesList.innerHTML = messagesList.innerHTML.replace(
/\[\[dyad-dump-path=([^\]]+)\]\]/g,
"[[dyad-dump-path=*]]",
);
});
}
await expect(this.page.getByTestId("messages-list")).toMatchAriaSnapshot({
timeout,
});
}
async snapshotServerDump(
type: "all-messages" | "last-message" | "request" = "all-messages",
{ name = "", dumpIndex = -1 }: { name?: string; dumpIndex?: number } = {},
) {
await this.waitForChatCompletion();
// Get the text content of the messages list
const messagesListText = await this.page
.getByTestId("messages-list")
.textContent();
// Find ALL dump paths using global regex
const dumpPathMatches = messagesListText?.match(
/\[\[dyad-dump-path=([^\]]+)\]\]/g,
);
if (!dumpPathMatches || dumpPathMatches.length === 0) {
throw new Error("No dump path found in messages list");
}
// Extract the actual paths from the matches
const dumpPaths = dumpPathMatches
.map((match) => {
const pathMatch = match.match(/\[\[dyad-dump-path=([^\]]+)\]\]/);
return pathMatch ? pathMatch[1] : null;
})
.filter(Boolean);
// Select the dump path based on index
// -1 means last, -2 means second to last, etc.
// 0 means first, 1 means second, etc.
const selectedIndex =
dumpIndex < 0 ? dumpPaths.length + dumpIndex : dumpIndex;
if (selectedIndex < 0 || selectedIndex >= dumpPaths.length) {
throw new Error(
`Dump index ${dumpIndex} is out of range. Found ${dumpPaths.length} dump paths.`,
);
}
const dumpFilePath = dumpPaths[selectedIndex];
if (!dumpFilePath) {
throw new Error("No dump file path found");
}
// Read the JSON file
const dumpContent: string = (fs.readFileSync(dumpFilePath, "utf-8") as any)
.replaceAll(/\[\[dyad-dump-path=([^\]]+)\]\]/g, "[[dyad-dump-path=*]]")
// Stabilize compaction backup file paths embedded in message text
// e.g. .dyad/chats/1/compaction-2026-02-05T21-25-24-285Z.md
.replaceAll(
/\.dyad\/chats\/\d+\/compaction-[^\s"\\]+\.md/g,
"[[compaction-backup-path]]",
);
// Perform snapshot comparison
const parsedDump = JSON.parse(dumpContent);
if (type === "request") {
if (parsedDump["body"]["input"]) {
parsedDump["body"]["input"] = parsedDump["body"]["input"].map(
(input: any) => {
if (input.role === "system") {
input.content = "[[SYSTEM_MESSAGE]]";
}
return input;
},
);
}
if (parsedDump["body"]["messages"]) {
parsedDump["body"]["messages"] = parsedDump["body"]["messages"].map(
(message: any) => {
if (message.role === "system") {
message.content = "[[SYSTEM_MESSAGE]]";
}
return message;
},
);
}
// Normalize fileIds to be deterministic based on content
normalizeVersionedFiles(parsedDump);
// Normalize item_reference IDs (e.g., msg_1234567890) to be deterministic
normalizeItemReferences(parsedDump);
// Normalize tool_call IDs (e.g., call_1234567890_0) to be deterministic
normalizeToolCallIds(parsedDump);
expect(
JSON.stringify(parsedDump, null, 2).replace(/\\r\\n/g, "\\n"),
).toMatchSnapshot(name);
return;
}
expect(
prettifyDump(
// responses API
parsedDump["body"]["input"] ??
// chat completion API
parsedDump["body"]["messages"],
{
onlyLastMessage: type === "last-message",
},
),
).toMatchSnapshot(name);
}
// ================================
// Test-only: Node.js Mock Control
// ================================
/**
* Set the mock state for Node.js installation status.
* @param installed - true = mock as installed, false = mock as not installed, null = use real check
*/
async setNodeMock(installed: boolean | null) {
await this.page.evaluate(async (installed) => {
await (window as any).electron.ipcRenderer.invoke("test:set-node-mock", {
installed,
});
}, installed);
}
}
/**
* Page object for agent tool consent banner.
* Handles consent interactions for agent tools.
*/
import { Page, expect } from "@playwright/test";
import { Timeout } from "../../constants";
export class AgentConsent {
constructor(public page: Page) {}
getAgentConsentBanner() {
return this.page
.getByRole("button", { name: "Always allow" })
.locator("..");
}
async waitForAgentConsentBanner(timeout = Timeout.MEDIUM) {
await expect(
this.page.getByRole("button", { name: "Always allow" }),
).toBeVisible({ timeout });
}
async clickAgentConsentAlwaysAllow() {
await this.page.getByRole("button", { name: "Always allow" }).click();
}
async clickAgentConsentAllowOnce() {
await this.page.getByRole("button", { name: "Allow once" }).click();
}
async clickAgentConsentDecline() {
await this.page.getByRole("button", { name: "Decline" }).click();
}
}
/**
* Page object for app management functionality.
* Handles app selection, importing, renaming, and app-related operations.
*/
import { Page, expect } from "@playwright/test";
import * as eph from "electron-playwright-helpers";
import { ElectronApplication } from "playwright";
import path from "path";
import { execSync, execFileSync } from "child_process";
import { Timeout } from "../../constants";
export class AppManagement {
constructor(
public page: Page,
private electronApp: ElectronApplication,
private userDataDir: string,
) {}
getTitleBarAppNameButton() {
return this.page.getByTestId("title-bar-app-name-button");
}
getAppListItem({ appName }: { appName: string }) {
return this.page.getByTestId(`app-list-item-${appName}`);
}
async isCurrentAppNameNone() {
await expect(async () => {
await expect(this.getTitleBarAppNameButton()).toContainText(
"no app selected",
);
}).toPass();
}
async getCurrentAppName() {
// Make sure to wait for the app to be set to avoid a race condition.
await expect(async () => {
await expect(this.getTitleBarAppNameButton()).not.toContainText(
"no app selected",
);
}).toPass();
return (await this.getTitleBarAppNameButton().textContent())?.replace(
"App: ",
"",
);
}
async getCurrentAppPath() {
const currentAppName = await this.getCurrentAppName();
if (!currentAppName) {
throw new Error("No current app name found");
}
return this.getAppPath({ appName: currentAppName });
}
getAppPath({ appName }: { appName: string }) {
return path.join(this.userDataDir, "dyad-apps", appName);
}
async clickAppListItem({ appName }: { appName: string }) {
await this.page.getByTestId(`app-list-item-${appName}`).click();
}
async clickOpenInChatButton() {
await this.page.getByRole("button", { name: "Open in Chat" }).click();
}
locateAppUpgradeButton({ upgradeId }: { upgradeId: string }) {
return this.page.getByTestId(`app-upgrade-${upgradeId}`);
}
async clickAppUpgradeButton({ upgradeId }: { upgradeId: string }) {
await this.locateAppUpgradeButton({ upgradeId }).click();
}
async expectAppUpgradeButtonIsNotVisible({
upgradeId,
}: {
upgradeId: string;
}) {
await expect(this.locateAppUpgradeButton({ upgradeId })).toBeHidden({
timeout: Timeout.MEDIUM,
});
}
async expectNoAppUpgrades() {
await expect(this.page.getByTestId("no-app-upgrades-needed")).toBeVisible({
timeout: Timeout.LONG,
});
}
async clickAppDetailsRenameAppButton() {
await this.page.getByTestId("app-details-rename-app-button").click();
}
async clickAppDetailsMoreOptions() {
await this.page.getByTestId("app-details-more-options-button").click();
}
async clickAppDetailsCopyAppButton() {
await this.page.getByRole("button", { name: "Copy app" }).click();
}
async clickConnectSupabaseButton() {
await this.page.getByTestId("connect-supabase-button").click();
}
async importApp(appDir: string) {
await this.page.getByRole("button", { name: "Import App" }).click();
await eph.stubDialog(this.electronApp, "showOpenDialog", {
filePaths: [
path.join(
__dirname,
"..",
"..",
"..",
"fixtures",
"import-app",
appDir,
),
],
});
await this.page.getByRole("button", { name: "Select Folder" }).click();
await this.page.getByRole("button", { name: "Import" }).click();
}
async configureGitUser({
email = "test@example.com",
name = "Test User",
disableGpgSign = true,
}: {
email?: string;
name?: string;
disableGpgSign?: boolean;
} = {}) {
const appPath = await this.getCurrentAppPath();
if (!appPath) {
throw new Error("App path not found");
}
execFileSync("git", ["config", "user.email", email], { cwd: appPath });
execFileSync("git", ["config", "user.name", name], { cwd: appPath });
if (disableGpgSign) {
execSync("git config commit.gpgsign false", { cwd: appPath });
}
}
async ensurePnpmInstall() {
const appPath = await this.getCurrentAppPath();
if (!appPath) {
throw new Error("No app selected");
}
const maxDurationMs = 180_000; // 3 minutes
const retryIntervalMs = 15_000;
const startTime = Date.now();
let lastOutput = "";
const checkCommand = `node -e 'const pkg=require("./package.json");const{execSync}=require("child_process");try{const prodResult=JSON.parse(execSync("pnpm list --json --depth=0",{encoding:"utf8"}));const devResult=JSON.parse(execSync("pnpm list --json --depth=0 --dev",{encoding:"utf8"}));const installed={...(prodResult[0]||{}).dependencies||{},...(devResult[0]||{}).devDependencies||{}};const expected=Object.keys({...pkg.dependencies||{},...pkg.devDependencies||{}});const missing=expected.filter(dep=>!installed[dep]);console.log(missing.length?"MISSING: "+missing.join(", "):"All dependencies installed")}catch(e){console.log("Error:",e.message)}'`;
while (Date.now() - startTime < maxDurationMs) {
try {
console.log(`Checking installed dependencies in ${appPath}...`);
const stdout = execSync(checkCommand, {
cwd: appPath,
stdio: "pipe",
encoding: "utf8",
});
lastOutput = (stdout || "").toString().trim();
console.log(`Dependency check output: ${lastOutput}`);
if (lastOutput.includes("All dependencies installed")) {
return;
}
} catch (error: any) {
// Capture any error output to include in the final error if we time out
const stdOut = error?.stdout ? error.stdout.toString() : "";
const stdErr = error?.stderr ? error.stderr.toString() : "";
lastOutput = [stdOut, stdErr, error?.message]
.filter(Boolean)
.join("\n");
console.error("Dependency check command failed:", lastOutput);
}
const elapsed = Date.now() - startTime;
const remaining = Math.max(0, maxDurationMs - elapsed);
const waitMs = Math.min(retryIntervalMs, remaining);
if (waitMs <= 0) break;
console.log(`Waiting ${waitMs}ms before retry...`);
await new Promise((resolve) => setTimeout(resolve, waitMs));
}
throw new Error(
`Dependencies not fully installed in ${appPath} after 3 minutes. Last output: ${lastOutput}`,
);
}
}
/**
* Page object for chat-related actions.
* Handles sending prompts, chat input, and chat mode selection.
*/
import { Page, expect } from "@playwright/test";
import { Timeout } from "../../constants";
export class ChatActions {
constructor(public page: Page) {}
getHomeChatInputContainer() {
return this.page.getByTestId("home-chat-input-container");
}
getChatInputContainer() {
return this.page.getByTestId("chat-input-container");
}
getChatInput() {
return this.page.locator(
'[data-lexical-editor="true"][aria-placeholder^="Ask Dyad to build"]',
);
}
/**
* Clears the Lexical chat input using keyboard shortcuts (Meta+A, Backspace).
* Uses toPass() for resilience since Lexical may need time to update its state.
*/
async clearChatInput() {
const chatInput = this.getChatInput();
await chatInput.click();
await this.page.keyboard.press("ControlOrMeta+a");
await this.page.keyboard.press("Backspace");
await expect(async () => {
const text = await chatInput.textContent();
expect(text?.trim()).toBe("");
}).toPass({ timeout: Timeout.SHORT });
}
/**
* Opens the chat history menu by clearing the input and pressing ArrowUp.
* Uses toPass() for resilience since the Lexical editor may need time to
* update its state before the history menu can be triggered.
*/
async openChatHistoryMenu() {
const historyMenu = this.page.locator('[data-mentions-menu="true"]');
await expect(async () => {
await this.clearChatInput();
await this.page.keyboard.press("ArrowUp");
await expect(historyMenu).toBeVisible({ timeout: 500 });
}).toPass({ timeout: Timeout.SHORT });
}
clickNewChat({ index = 0 }: { index?: number } = {}) {
// There is two new chat buttons...
return this.page
.getByRole("button", { name: "New Chat" })
.nth(index)
.click();
}
private getRetryButton() {
return this.page.getByRole("button", { name: "Retry" });
}
private getUndoButton() {
return this.page.getByRole("button", { name: "Undo" });
}
async waitForChatCompletion() {
await expect(this.getRetryButton()).toBeVisible({
timeout: Timeout.MEDIUM,
});
}
async clickRetry() {
await this.getRetryButton().click();
}
async clickUndo() {
await this.getUndoButton().click();
}
async sendPrompt(
prompt: string,
{ skipWaitForCompletion = false }: { skipWaitForCompletion?: boolean } = {},
) {
await this.getChatInput().click();
await this.getChatInput().fill(prompt);
await this.page.getByRole("button", { name: "Send message" }).click();
if (!skipWaitForCompletion) {
await this.waitForChatCompletion();
}
}
async selectChatMode(
mode: "build" | "ask" | "agent" | "local-agent" | "basic-agent" | "plan",
) {
await this.page.getByTestId("chat-mode-selector").click();
const mapping: Record<string, string> = {
build: "Build Generate and edit code",
ask: "Ask Ask",
agent: "Build with MCP",
"local-agent": "Agent v2",
"basic-agent": "Basic Agent", // For free users
plan: "Plan.*Design before you build",
};
const optionName = mapping[mode];
await this.page
.getByRole("option", {
name: new RegExp(optionName),
})
.click();
}
async selectLocalAgentMode() {
await this.selectChatMode("local-agent");
}
async clickChatActivityButton() {
await this.page.getByTestId("chat-activity-button").click();
}
async snapshotChatActivityList() {
await expect(
this.page.getByTestId("chat-activity-list"),
).toMatchAriaSnapshot();
}
async snapshotChatInputContainer() {
await expect(this.getChatInputContainer()).toMatchAriaSnapshot();
}
}
/**
* Page object for the inline code editor.
* Handles editing, saving, and canceling file edits.
*/
import { Page } from "@playwright/test";
export class CodeEditor {
constructor(public page: Page) {}
async clickEditButton() {
await this.page.locator('button:has-text("Edit")').first().click();
}
async editFileContent(content: string) {
const editor = this.page.locator(".monaco-editor textarea").first();
await editor.focus();
await editor.press("Home");
await editor.type(content);
}
async saveFile() {
await this.page.locator('[data-testid="save-file-button"]').click();
}
async cancelEdit() {
await this.page.locator('button:has-text("Cancel")').first().click();
}
}
/**
* Page object for GitHub integration testing.
* Handles connecting, creating/syncing repos, and verifying push events.
*/
import { Page, expect } from "@playwright/test";
export class GitHubConnector {
constructor(public page: Page) {}
async connect() {
await this.page.getByRole("button", { name: "Connect to GitHub" }).click();
}
getSetupYourGitHubRepoButton() {
return this.page.getByText("Set up your GitHub repo");
}
getCreateNewRepoModeButton() {
return this.page.getByRole("button", { name: "Create new repo" });
}
getConnectToExistingRepoModeButton() {
return this.page.getByRole("button", { name: "Connect to existing repo" });
}
async clickCreateRepoButton() {
await this.page.getByRole("button", { name: "Create Repo" }).click();
}
async fillCreateRepoName(name: string) {
await this.page.getByTestId("github-create-repo-name-input").fill(name);
}
async fillNewRepoBranchName(name: string) {
await this.page.getByTestId("github-new-repo-branch-input").fill(name);
}
async selectRepo(repo: string) {
await this.page.getByTestId("github-repo-select").click();
await this.page.getByRole("option", { name: repo }).click();
}
async selectBranch(branch: string) {
await this.page.getByTestId("github-branch-select").click();
await this.page.getByRole("option", { name: branch }).click();
}
async selectCustomBranch(branch: string) {
await this.page.getByTestId("github-branch-select").click();
await this.page
.getByRole("option", { name: "✏️ Type custom branch name" })
.click();
await this.page.getByTestId("github-custom-branch-input").click();
await this.page.getByTestId("github-custom-branch-input").fill(branch);
}
async clickConnectToRepoButton() {
await this.page.getByRole("button", { name: "Connect to repo" }).click();
}
async snapshotConnectedRepo() {
await expect(
this.page.getByTestId("github-connected-repo"),
).toMatchAriaSnapshot();
}
async snapshotSetupRepo() {
await expect(
this.page.getByTestId("github-setup-repo"),
).toMatchAriaSnapshot();
}
async snapshotUnconnectedRepo() {
await expect(
this.page.getByTestId("github-unconnected-repo"),
).toMatchAriaSnapshot();
}
async clickSyncToGithubButton() {
await this.page.getByRole("button", { name: "Sync to GitHub" }).click();
}
async clickDisconnectRepoButton() {
await this.page
.getByRole("button", { name: "Disconnect from repo" })
.click();
}
async clearPushEvents() {
const response = await this.page.request.post(
"http://localhost:3500/github/api/test/clear-push-events",
);
return await response.json();
}
async getPushEvents(repo?: string) {
const url = repo
? `http://localhost:3500/github/api/test/push-events?repo=${repo}`
: "http://localhost:3500/github/api/test/push-events";
const response = await this.page.request.get(url);
return await response.json();
}
async verifyPushEvent(expectedEvent: {
repo: string;
branch: string;
operation?: "push" | "create" | "delete";
}) {
const pushEvents = await this.getPushEvents(expectedEvent.repo);
const matchingEvent = pushEvents.find(
(event: any) =>
event.repo === expectedEvent.repo &&
event.branch === expectedEvent.branch &&
(!expectedEvent.operation ||
event.operation === expectedEvent.operation),
);
if (!matchingEvent) {
throw new Error(
`Expected push event not found. Expected: ${JSON.stringify(expectedEvent)}. ` +
`Actual events: ${JSON.stringify(pushEvents)}`,
);
}
return matchingEvent;
}
}
/**
* Page object for model picker functionality.
* Handles model and provider selection.
*/
import { Page } from "@playwright/test";
export class ModelPicker {
constructor(public page: Page) {}
async selectModel({ provider, model }: { provider: string; model: string }) {
await this.page.getByTestId("model-picker").click();
await this.page.getByText(provider, { exact: true }).click();
await this.page.getByText(model, { exact: true }).click();
}
async selectTestModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("test-provider").click();
await this.page.getByText("test-model").click();
}
async selectTestOllamaModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Local models").click();
await this.page.getByText("Ollama", { exact: true }).click();
await this.page.getByText("Testollama", { exact: true }).click();
}
async selectTestLMStudioModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Local models").click();
await this.page.getByText("LM Studio", { exact: true }).click();
// Both of the elements that match "lmstudio-model-1" are the same button, so we just pick the first.
await this.page
.getByText("lmstudio-model-1", { exact: true })
.first()
.click();
}
async selectTestAzureModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Other AI providers").click();
await this.page.getByText("Azure OpenAI", { exact: true }).click();
await this.page.getByText("GPT-5", { exact: true }).click();
}
}
/**
* Page object for navigation between tabs and pages.
* Handles tab navigation and back button.
*/
import { Page, expect } from "@playwright/test";
export class Navigation {
constructor(public page: Page) {}
async goToSettingsTab() {
await this.page.getByRole("link", { name: "Settings" }).click();
}
async goToLibraryTab() {
await this.page.getByRole("link", { name: "Library" }).click();
}
async goToAppsTab() {
await this.page.getByRole("link", { name: "Apps" }).click();
await expect(this.page.getByText("Build a new app")).toBeVisible();
}
async goToChatTab() {
await this.page.getByRole("link", { name: "Chat" }).click();
}
async goToHubTab() {
await this.page.getByRole("link", { name: "Hub" }).click();
}
async clickBackButton() {
await this.page.getByRole("button", { name: "Back" }).click();
}
async selectTemplate(templateName: string) {
await this.page.getByRole("img", { name: templateName }).click();
}
async goToHubAndSelectTemplate(templateName: "Next.js Template") {
await this.goToHubTab();
await this.selectTemplate(templateName);
await this.goToAppsTab();
}
}
/**
* Page object for the preview panel.
* Handles preview mode selection, iframe interactions, and error handling.
*/
import { Page, expect } from "@playwright/test";
import { Timeout } from "../../constants";
export class PreviewPanel {
constructor(public page: Page) {}
async selectPreviewMode(
mode:
| "code"
| "problems"
| "preview"
| "configure"
| "security"
| "publish",
) {
await this.page.getByTestId(`${mode}-mode-button`).click();
}
async clickRecheckProblems() {
await this.page.getByTestId("recheck-button").click();
}
async clickFixAllProblems() {
await this.page.getByTestId("fix-all-button").click();
}
async snapshotProblemsPane() {
await expect(this.page.getByTestId("problems-pane")).toMatchAriaSnapshot({
timeout: Timeout.MEDIUM,
});
}
async clickRebuild() {
await this.clickPreviewMoreOptions();
await this.page.getByText("Rebuild").click();
}
async clickTogglePreviewPanel() {
await this.page.getByTestId("toggle-preview-panel-button").click();
}
async clickPreviewPickElement() {
await this.page
.getByTestId("preview-pick-element-button")
.click({ timeout: Timeout.EXTRA_LONG });
}
async clickDeselectComponent(options?: { index?: number }) {
const buttons = this.page.getByRole("button", {
name: "Deselect component",
});
if (options?.index !== undefined) {
await buttons.nth(options.index).click();
} else {
await buttons.first().click();
}
}
async clickPreviewMoreOptions() {
await this.page.getByTestId("preview-more-options-button").click();
}
async clickPreviewRefresh() {
await this.page.getByTestId("preview-refresh-button").click();
}
async clickPreviewNavigateBack() {
await this.page.getByTestId("preview-navigate-back-button").click();
}
async clickPreviewNavigateForward() {
await this.page.getByTestId("preview-navigate-forward-button").click();
}
async clickPreviewOpenBrowser() {
await this.page.getByTestId("preview-open-browser-button").click();
}
async clickPreviewAnnotatorButton() {
await this.page
.getByTestId("preview-annotator-button")
.click({ timeout: Timeout.EXTRA_LONG });
}
async waitForAnnotatorMode() {
// Wait for the annotator toolbar to be visible
await expect(this.page.getByRole("button", { name: "Select" })).toBeVisible(
{
timeout: Timeout.MEDIUM,
},
);
}
async clickAnnotatorSubmit() {
await this.page.getByRole("button", { name: "Add to Chat" }).click();
}
locateLoadingAppPreview() {
return this.page.getByText("Preparing app preview...");
}
locateStartingAppPreview() {
return this.page.getByText("Starting your app server...");
}
getPreviewIframeElement() {
return this.page.getByTestId("preview-iframe-element");
}
expectPreviewIframeIsVisible() {
return expect(this.getPreviewIframeElement()).toBeVisible({
timeout: Timeout.LONG,
});
}
async clickFixErrorWithAI() {
await this.page.getByRole("button", { name: "Fix error with AI" }).click();
}
async clickCopyErrorMessage() {
await this.page
.getByTestId("preview-error-banner")
.getByRole("button", { name: /Copy/ })
.click();
}
async clickFixAllErrors() {
await this.page.getByRole("button", { name: /Fix All Errors/ }).click();
}
async snapshotPreviewErrorBanner() {
await expect(this.locatePreviewErrorBanner()).toMatchAriaSnapshot({
timeout: Timeout.LONG,
});
}
locatePreviewErrorBanner() {
return this.page.getByTestId("preview-error-banner");
}
getSelectedComponentsDisplay() {
return this.page.getByTestId("selected-component-display");
}
async snapshotSelectedComponentsDisplay() {
await expect(this.getSelectedComponentsDisplay()).toMatchAriaSnapshot();
}
async snapshotPreview({ name }: { name?: string } = {}) {
const iframe = this.getPreviewIframeElement();
await expect(iframe.contentFrame().locator("body")).toMatchAriaSnapshot({
name,
timeout: Timeout.LONG,
});
}
}
/**
* Page object for prompt library functionality.
* Handles creating and managing prompts.
*/
import { Page } from "@playwright/test";
export class PromptLibrary {
constructor(public page: Page) {}
async createPrompt({
title,
description,
content,
}: {
title: string;
description?: string;
content: string;
}) {
await this.page.getByRole("button", { name: "New Prompt" }).click();
await this.page.getByRole("textbox", { name: "Title" }).fill(title);
if (description) {
await this.page
.getByRole("textbox", { name: "Description (optional)" })
.fill(description);
}
await this.page.getByRole("textbox", { name: "Content" }).fill(content);
await this.page.getByRole("button", { name: "Save" }).click();
}
}
/**
* Page object for security review functionality.
* Handles running security reviews and managing findings.
*/
import { Page, expect } from "@playwright/test";
import { ChatActions } from "./ChatActions";
export class SecurityReview {
private chatActions: ChatActions;
constructor(public page: Page) {
this.chatActions = new ChatActions(page);
}
async clickRunSecurityReview() {
const runSecurityReviewButton = this.page
.getByRole("button", { name: "Run Security Review" })
.first();
await runSecurityReviewButton.click();
await runSecurityReviewButton.waitFor({ state: "hidden" });
await this.chatActions.waitForChatCompletion();
}
async snapshotSecurityFindingsTable() {
await expect(
this.page.getByTestId("security-findings-table"),
).toMatchAriaSnapshot();
}
}
/**
* Page object for settings functionality.
* Handles toggles, settings recording, and provider configuration.
*/
import { Page, expect } from "@playwright/test";
import fs from "fs";
import path from "path";
export class Settings {
constructor(
public page: Page,
private userDataDir: string,
) {}
async toggleAutoApprove() {
await this.page.getByRole("switch", { name: "Auto-approve" }).click();
}
async toggleLocalAgentMode() {
await this.page.getByRole("switch", { name: "Enable Agent v2" }).click();
}
async toggleNativeGit() {
await this.page.getByRole("switch", { name: "Enable Native Git" }).click();
}
async toggleAutoFixProblems() {
await this.page.getByRole("switch", { name: "Auto-fix problems" }).click();
}
async toggleAutoUpdate() {
await this.page.getByRole("switch", { name: "Auto-update" }).click();
}
async changeReleaseChannel(channel: "stable" | "beta") {
await this.page.getByRole("combobox", { name: "Release Channel" }).click();
await this.page
.getByRole("option", { name: channel === "stable" ? "Stable" : "Beta" })
.click();
}
async clickTelemetryAccept() {
await this.page.getByTestId("telemetry-accept-button").click();
}
async clickTelemetryReject() {
await this.page.getByTestId("telemetry-reject-button").click();
}
async clickTelemetryLater() {
await this.page.getByTestId("telemetry-later-button").click();
}
/**
* Records the current settings state for later comparison.
* Use with `snapshotSettingsDelta()` to snapshot only what changed.
*/
recordSettings(): Record<string, unknown> {
const settingsPath = path.join(this.userDataDir, "user-settings.json");
const settingsContent = fs.readFileSync(settingsPath, "utf-8");
return JSON.parse(settingsContent);
}
/**
* Snapshots only the differences between the current settings and a previously recorded state.
* Output is in git diff style for easy reading.
*/
snapshotSettingsDelta(beforeSettings: Record<string, unknown>) {
const afterSettings = this.recordSettings();
const diffLines: string[] = [];
const allKeys = new Set([
...Object.keys(beforeSettings),
...Object.keys(afterSettings),
]);
// Sort keys for deterministic output
const sortedKeys = Array.from(allKeys).sort();
// Keys whose values should be redacted for deterministic snapshots
const redactedKeys: Record<string, string> = {
telemetryUserId: "[UUID]",
lastShownReleaseNotesVersion: "[scrubbed]",
};
for (const key of sortedKeys) {
const beforeValue = beforeSettings[key];
const afterValue = afterSettings[key];
const beforeExists = key in beforeSettings;
const afterExists = key in afterSettings;
// Format value with diff marker on each line for multiline values
// Redact certain keys for deterministic snapshots
const formatValue = (val: unknown, marker: "+" | "-") => {
const displayVal = key in redactedKeys ? redactedKeys[key] : val;
const lines = JSON.stringify(displayVal, null, 2).split("\n");
return lines
.map((line, i) => (i === 0 ? line : `${marker} ${line}`))
.join("\n");
};
if (!beforeExists && afterExists) {
// Added
diffLines.push(`+ "${key}": ${formatValue(afterValue, "+")}`);
} else if (beforeExists && !afterExists) {
// Removed
diffLines.push(`- "${key}": ${formatValue(beforeValue, "-")}`);
} else if (JSON.stringify(beforeValue) !== JSON.stringify(afterValue)) {
// Changed
diffLines.push(`- "${key}": ${formatValue(beforeValue, "-")}`);
diffLines.push(`+ "${key}": ${formatValue(afterValue, "+")}`);
}
}
expect(diffLines.join("\n")).toMatchSnapshot();
}
async setUpTestProvider() {
await this.page.getByText("Add custom providerConnect to").click();
// Fill out provider dialog
await this.page
.getByRole("textbox", { name: "Provider ID" })
.fill("testing");
await this.page.getByRole("textbox", { name: "Display Name" }).click();
await this.page
.getByRole("textbox", { name: "Display Name" })
.fill("test-provider");
await this.page.getByText("API Base URLThe base URL for").click();
await this.page
.getByRole("textbox", { name: "API Base URL" })
.fill("http://localhost:3500/v1");
await this.page.getByRole("button", { name: "Add Provider" }).click();
}
async setUpTestModel() {
await this.page.getByRole("heading", { name: "test-provider" }).click();
await this.page.getByRole("button", { name: "Add Custom Model" }).click();
await this.page
.getByRole("textbox", { name: "Model ID*" })
.fill("test-model");
await this.page.getByRole("textbox", { name: "Model ID*" }).press("Tab");
await this.page.getByRole("textbox", { name: "Name*" }).fill("test-model");
await this.page.getByRole("button", { name: "Add Model" }).click();
}
async addCustomTestModel({
name,
contextWindow,
}: {
name: string;
contextWindow?: number;
}) {
await this.page.getByRole("heading", { name: "test-provider" }).click();
await this.page.getByRole("button", { name: "Add Custom Model" }).click();
await this.page.getByRole("textbox", { name: "Model ID*" }).fill(name);
await this.page.getByRole("textbox", { name: "Model ID*" }).press("Tab");
await this.page.getByRole("textbox", { name: "Name*" }).fill(name);
if (contextWindow) {
await this.page.locator("#context-window").fill(String(contextWindow));
}
await this.page.getByRole("button", { name: "Add Model" }).click();
}
async setUpTestProviderApiKey() {
// Fill in a test API key for the custom provider
await this.page
.getByPlaceholder(/Enter new.*API Key here/)
.fill("test-api-key-12345");
await this.page.getByRole("button", { name: "Save Key" }).click();
// Wait for the key to be saved
await expect(this.page.getByText("test-api-key-12345")).toBeVisible();
}
async setUpDyadProvider() {
await this.page
.locator("div")
.filter({ hasText: /^DyadNeeds Setup$/ })
.nth(1)
.click();
await this.page.getByRole("textbox", { name: "Set Dyad API Key" }).click();
await this.page
.getByRole("textbox", { name: "Set Dyad API Key" })
.fill("testdyadkey");
await this.page.getByRole("button", { name: "Save Key" }).click();
}
}
/**
* Page object for toast notifications.
* Handles waiting for, asserting, and dismissing toasts.
*/
import { Page, expect } from "@playwright/test";
import { Timeout } from "../../constants";
export class ToastNotifications {
constructor(public page: Page) {}
async expectNoToast() {
await expect(this.page.locator("[data-sonner-toast]")).toHaveCount(0);
}
async waitForToast(
type?: "success" | "error" | "warning" | "info",
timeout = 5000,
) {
const selector = type
? `[data-sonner-toast][data-type="${type}"]`
: "[data-sonner-toast]";
await this.page.waitForSelector(selector, { timeout });
}
async waitForToastWithText(text: string, timeout = Timeout.MEDIUM) {
await this.page.waitForSelector(`[data-sonner-toast]:has-text("${text}")`, {
timeout,
});
}
async assertToastVisible(type?: "success" | "error" | "warning" | "info") {
const selector = type
? `[data-sonner-toast][data-type="${type}"]`
: "[data-sonner-toast]";
await expect(this.page.locator(selector)).toBeVisible();
}
async assertToastWithText(text: string) {
await expect(
this.page.locator(`[data-sonner-toast]:has-text("${text}")`),
).toBeVisible();
}
async dismissAllToasts() {
// Click all close buttons if they exist
const closeButtons = this.page.locator(
"[data-sonner-toast] button[data-close-button]",
);
const maxAttempts = 20;
let attempts = 0;
while ((await closeButtons.count()) > 0 && attempts < maxAttempts) {
await closeButtons
.first()
.click()
.catch(() => {});
attempts++;
}
}
}
/**
* Barrel file for component page objects.
*/
export { GitHubConnector } from "./GitHubConnector";
export { ChatActions } from "./ChatActions";
export { PreviewPanel } from "./PreviewPanel";
export { CodeEditor } from "./CodeEditor";
export { SecurityReview } from "./SecurityReview";
export { ToastNotifications } from "./ToastNotifications";
export { AgentConsent } from "./AgentConsent";
export { Navigation } from "./Navigation";
export { ModelPicker } from "./ModelPicker";
export { Settings } from "./Settings";
export { AppManagement } from "./AppManagement";
export { PromptLibrary } from "./PromptLibrary";
/**
* Page object for the Context Files Picker dialog.
* Handles adding and removing context files in tests.
*/
import { Page } from "@playwright/test";
export class ContextFilesPickerDialog {
constructor(
public page: Page,
public close: () => Promise<void>,
) {}
async addManualContextFile(path: string) {
await this.page.getByTestId("manual-context-files-input").fill(path);
await this.page.getByTestId("manual-context-files-add-button").click();
}
async addAutoIncludeContextFile(path: string) {
await this.page.getByTestId("auto-include-context-files-input").fill(path);
await this.page
.getByTestId("auto-include-context-files-add-button")
.click();
}
async removeManualContextFile() {
await this.page
.getByTestId("manual-context-files-remove-button")
.first()
.click();
}
async removeAutoIncludeContextFile() {
await this.page
.getByTestId("auto-include-context-files-remove-button")
.first()
.click();
}
async addExcludeContextFile(path: string) {
await this.page.getByTestId("exclude-context-files-input").fill(path);
await this.page.getByTestId("exclude-context-files-add-button").click();
}
async removeExcludeContextFile() {
await this.page
.getByTestId("exclude-context-files-remove-button")
.first()
.click();
}
}
/**
* Page object for the Pro Modes dialog.
* Handles smart context and turbo edits mode selection.
*/
import { Page } from "@playwright/test";
export class ProModesDialog {
constructor(
public page: Page,
public close: () => Promise<void>,
) {}
async setSmartContextMode(mode: "balanced" | "off" | "deep") {
await this.page
.getByTestId("smart-context-selector")
.getByRole("button", {
name: mode.charAt(0).toUpperCase() + mode.slice(1),
})
.click();
}
async setTurboEditsMode(mode: "off" | "classic" | "search-replace") {
await this.page
.getByTestId("turbo-edits-selector")
.getByRole("button", {
name:
mode === "search-replace"
? "Search & replace"
: mode.charAt(0).toUpperCase() + mode.slice(1),
})
.click();
}
}
/**
* Barrel file for dialog page objects.
*/
export { ContextFilesPickerDialog } from "./ContextFilesPickerDialog";
export { ProModesDialog } from "./ProModesDialog";
/**
* Barrel file for page objects.
*/
// Main page object
export { PageObject } from "./PageObject";
// Dialog page objects
export { ContextFilesPickerDialog, ProModesDialog } from "./dialogs";
// Component page objects
export {
GitHubConnector,
ChatActions,
PreviewPanel,
CodeEditor,
SecurityReview,
ToastNotifications,
AgentConsent,
Navigation,
ModelPicker,
Settings,
AppManagement,
PromptLibrary,
} from "./components";
import { test as base, Page, expect } from "@playwright/test";
import * as eph from "electron-playwright-helpers";
import { ElectronApplication, _electron as electron } from "playwright";
import fs from "fs";
import path from "path";
import os from "os";
import { execSync } from "child_process";
import { generateAppFilesSnapshotData } from "./generateAppFilesSnapshotData";
import {
BUILD_SYSTEM_POSTFIX,
BUILD_SYSTEM_PREFIX,
} from "@/prompts/system_prompt";
const showDebugLogs = process.env.DEBUG_LOGS === "true";
export const Timeout = {
// Things generally take longer on CI, so we make them longer.
EXTRA_LONG: process.env.CI ? 120_000 : 60_000,
LONG: process.env.CI ? 60_000 : 30_000,
MEDIUM: process.env.CI ? 30_000 : 15_000,
SHORT: process.env.CI ? 5_000 : 2_000,
};
/**
* Normalizes item_reference IDs in the input array to be deterministic.
* item_reference objects have the shape { type: "item_reference", id: "msg_..." }
* where the ID is a timestamp-based value that changes between test runs.
*/
function normalizeItemReferences(dump: any): void {
const input = dump?.body?.input;
if (!Array.isArray(input)) {
return;
}
let refIndex = 0;
for (const item of input) {
if (item?.type === "item_reference" && item?.id) {
item.id = `[[ITEM_REF_${refIndex}]]`;
refIndex++;
}
}
}
/** /**
* Normalizes tool_call IDs and tool_call_id references to be deterministic. * Main test helper module for e2e tests.
* Tool call IDs have the format "call_[timestamp]_[index]" which changes between runs. *
* This file re-exports all testing utilities for backward compatibility.
* The actual implementations have been modularized into separate files:
*
* - constants.ts: Timeout constants and configuration
* - fixtures.ts: Playwright test fixtures and configuration helpers
* - utils/: Normalization and dump prettification utilities
* - page-objects/: Page object classes organized by component
* - PageObject.ts: Main page object composing all components
* - components/: Individual component page objects
* - dialogs/: Dialog-specific page objects
*/ */
function normalizeToolCallIds(dump: any): void {
const messages = dump?.body?.messages;
if (!Array.isArray(messages)) {
return;
}
const oldToNewId: Record<string, string> = {};
let toolCallIndex = 0;
// First pass: collect all tool_call IDs and create mapping
for (const message of messages) {
if (message?.tool_calls && Array.isArray(message.tool_calls)) {
for (const toolCall of message.tool_calls) {
if (toolCall?.id && !oldToNewId[toolCall.id]) {
oldToNewId[toolCall.id] = `[[TOOL_CALL_${toolCallIndex}]]`;
toolCallIndex++;
}
}
}
}
// Second pass: replace all IDs
for (const message of messages) {
if (message?.tool_calls && Array.isArray(message.tool_calls)) {
for (const toolCall of message.tool_calls) {
if (toolCall?.id && oldToNewId[toolCall.id]) {
toolCall.id = oldToNewId[toolCall.id];
}
}
}
if (message?.tool_call_id && oldToNewId[message.tool_call_id]) {
message.tool_call_id = oldToNewId[message.tool_call_id];
}
}
}
/**
* Normalizes fileId hashes in versioned_files to be deterministic.
* FileIds are SHA-256 hashes that may include non-deterministic components
* like app paths with timestamps. This replaces them with stable placeholders
* based on content sorting.
*/
function normalizeVersionedFiles(dump: any): void {
const vf = dump?.body?.dyad_options?.versioned_files;
if (!vf?.fileIdToContent) {
return;
}
const fileIdToContent = vf.fileIdToContent as Record<string, string>;
// Create mapping from old fileId to new deterministic fileId
// Sort by content to ensure deterministic ordering
const entries = Object.entries(fileIdToContent).sort((a, b) =>
String(a[1]).localeCompare(String(b[1])),
);
const oldToNewId: Record<string, string> = {};
const newFileIdToContent: Record<string, string> = {};
entries.forEach(([oldId, content], index) => {
const newId = `[[FILE_ID_${index}]]`;
oldToNewId[oldId] = newId;
newFileIdToContent[newId] = content;
});
vf.fileIdToContent = newFileIdToContent;
// Update fileReferences
if (vf.fileReferences) {
vf.fileReferences = vf.fileReferences.map((ref: any) => ({
...ref,
fileId: oldToNewId[ref.fileId] ?? ref.fileId,
}));
}
// Update messageIndexToFilePathToFileId
if (vf.messageIndexToFilePathToFileId) {
for (const pathToId of Object.values(
vf.messageIndexToFilePathToFileId as Record<
string,
Record<string, string>
>,
)) {
for (const [filePath, id] of Object.entries(pathToId)) {
pathToId[filePath] = oldToNewId[id] ?? id;
}
}
}
}
export class ContextFilesPickerDialog {
constructor(
public page: Page,
public close: () => Promise<void>,
) {}
async addManualContextFile(path: string) {
await this.page.getByTestId("manual-context-files-input").fill(path);
await this.page.getByTestId("manual-context-files-add-button").click();
}
async addAutoIncludeContextFile(path: string) {
await this.page.getByTestId("auto-include-context-files-input").fill(path);
await this.page
.getByTestId("auto-include-context-files-add-button")
.click();
}
async removeManualContextFile() {
await this.page
.getByTestId("manual-context-files-remove-button")
.first()
.click();
}
async removeAutoIncludeContextFile() {
await this.page
.getByTestId("auto-include-context-files-remove-button")
.first()
.click();
}
async addExcludeContextFile(path: string) {
await this.page.getByTestId("exclude-context-files-input").fill(path);
await this.page.getByTestId("exclude-context-files-add-button").click();
}
async removeExcludeContextFile() {
await this.page
.getByTestId("exclude-context-files-remove-button")
.first()
.click();
}
}
class ProModesDialog {
constructor(
public page: Page,
public close: () => Promise<void>,
) {}
async setSmartContextMode(mode: "balanced" | "off" | "deep") {
await this.page
.getByTestId("smart-context-selector")
.getByRole("button", {
name: mode.charAt(0).toUpperCase() + mode.slice(1),
})
.click();
}
async setTurboEditsMode(mode: "off" | "classic" | "search-replace") {
await this.page
.getByTestId("turbo-edits-selector")
.getByRole("button", {
name:
mode === "search-replace"
? "Search & replace"
: mode.charAt(0).toUpperCase() + mode.slice(1),
})
.click();
}
}
class GitHubConnector {
constructor(public page: Page) {}
async connect() {
await this.page.getByRole("button", { name: "Connect to GitHub" }).click();
}
getSetupYourGitHubRepoButton() {
return this.page.getByText("Set up your GitHub repo");
}
getCreateNewRepoModeButton() {
return this.page.getByRole("button", { name: "Create new repo" });
}
getConnectToExistingRepoModeButton() {
return this.page.getByRole("button", { name: "Connect to existing repo" });
}
async clickCreateRepoButton() {
await this.page.getByRole("button", { name: "Create Repo" }).click();
}
async fillCreateRepoName(name: string) {
await this.page.getByTestId("github-create-repo-name-input").fill(name);
}
async fillNewRepoBranchName(name: string) {
await this.page.getByTestId("github-new-repo-branch-input").fill(name);
}
async selectRepo(repo: string) {
await this.page.getByTestId("github-repo-select").click();
await this.page.getByRole("option", { name: repo }).click();
}
async selectBranch(branch: string) {
await this.page.getByTestId("github-branch-select").click();
await this.page.getByRole("option", { name: branch }).click();
}
async selectCustomBranch(branch: string) {
await this.page.getByTestId("github-branch-select").click();
await this.page
.getByRole("option", { name: "✏️ Type custom branch name" })
.click();
await this.page.getByTestId("github-custom-branch-input").click();
await this.page.getByTestId("github-custom-branch-input").fill(branch);
}
async clickConnectToRepoButton() {
await this.page.getByRole("button", { name: "Connect to repo" }).click();
}
async snapshotConnectedRepo() {
await expect(
this.page.getByTestId("github-connected-repo"),
).toMatchAriaSnapshot();
}
async snapshotSetupRepo() {
await expect(
this.page.getByTestId("github-setup-repo"),
).toMatchAriaSnapshot();
}
async snapshotUnconnectedRepo() {
await expect(
this.page.getByTestId("github-unconnected-repo"),
).toMatchAriaSnapshot();
}
async clickSyncToGithubButton() {
await this.page.getByRole("button", { name: "Sync to GitHub" }).click();
}
async clickDisconnectRepoButton() {
await this.page
.getByRole("button", { name: "Disconnect from repo" })
.click();
}
async clearPushEvents() {
const response = await this.page.request.post(
"http://localhost:3500/github/api/test/clear-push-events",
);
return await response.json();
}
async getPushEvents(repo?: string) {
const url = repo
? `http://localhost:3500/github/api/test/push-events?repo=${repo}`
: "http://localhost:3500/github/api/test/push-events";
const response = await this.page.request.get(url);
return await response.json();
}
async verifyPushEvent(expectedEvent: {
repo: string;
branch: string;
operation?: "push" | "create" | "delete";
}) {
const pushEvents = await this.getPushEvents(expectedEvent.repo);
const matchingEvent = pushEvents.find(
(event: any) =>
event.repo === expectedEvent.repo &&
event.branch === expectedEvent.branch &&
(!expectedEvent.operation ||
event.operation === expectedEvent.operation),
);
if (!matchingEvent) {
throw new Error(
`Expected push event not found. Expected: ${JSON.stringify(expectedEvent)}. ` +
`Actual events: ${JSON.stringify(pushEvents)}`,
);
}
return matchingEvent;
}
}
export class PageObject {
public userDataDir: string;
public githubConnector: GitHubConnector;
constructor(
public electronApp: ElectronApplication,
public page: Page,
{ userDataDir }: { userDataDir: string },
) {
this.userDataDir = userDataDir;
this.githubConnector = new GitHubConnector(this.page);
}
private async baseSetup() {
await this.githubConnector.clearPushEvents();
}
async setUp({
autoApprove = false,
disableNativeGit = false,
enableAutoFixProblems = false,
enableBasicAgent = false,
}: {
autoApprove?: boolean;
disableNativeGit?: boolean;
enableAutoFixProblems?: boolean;
enableBasicAgent?: boolean;
} = {}) {
await this.baseSetup();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
if (disableNativeGit) {
await this.toggleNativeGit();
}
if (enableAutoFixProblems) {
await this.toggleAutoFixProblems();
}
await this.setUpTestProvider();
await this.setUpTestModel();
await this.goToAppsTab();
if (!enableBasicAgent) {
await this.selectChatMode("build");
}
await this.selectTestModel();
}
async setUpDyadPro({
autoApprove = false,
localAgent = false,
localAgentUseAutoModel = false,
}: {
autoApprove?: boolean;
localAgent?: boolean;
localAgentUseAutoModel?: boolean;
} = {}) {
await this.baseSetup();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
await this.setUpDyadProvider();
await this.goToAppsTab();
if (!localAgent) {
await this.selectChatMode("build");
}
// Select a non-openAI model for local agent mode,
// since openAI models go to the responses API.
if (localAgent && !localAgentUseAutoModel) {
await this.selectModel({
provider: "Anthropic",
model: "Claude Opus 4.5",
});
}
}
async ensurePnpmInstall() {
const appPath = await this.getCurrentAppPath();
if (!appPath) {
throw new Error("No app selected");
}
const maxDurationMs = 180_000; // 3 minutes
const retryIntervalMs = 15_000;
const startTime = Date.now();
let lastOutput = "";
const checkCommand = `node -e 'const pkg=require("./package.json");const{execSync}=require("child_process");try{const prodResult=JSON.parse(execSync("pnpm list --json --depth=0",{encoding:"utf8"}));const devResult=JSON.parse(execSync("pnpm list --json --depth=0 --dev",{encoding:"utf8"}));const installed={...(prodResult[0]||{}).dependencies||{},...(devResult[0]||{}).devDependencies||{}};const expected=Object.keys({...pkg.dependencies||{},...pkg.devDependencies||{}});const missing=expected.filter(dep=>!installed[dep]);console.log(missing.length?"MISSING: "+missing.join(", "):"All dependencies installed")}catch(e){console.log("Error:",e.message)}'`;
while (Date.now() - startTime < maxDurationMs) {
try {
console.log(`Checking installed dependencies in ${appPath}...`);
const stdout = execSync(checkCommand, {
cwd: appPath,
stdio: "pipe",
encoding: "utf8",
});
lastOutput = (stdout || "").toString().trim();
console.log(`Dependency check output: ${lastOutput}`);
if (lastOutput.includes("All dependencies installed")) {
return;
}
} catch (error: any) {
// Capture any error output to include in the final error if we time out
const stdOut = error?.stdout ? error.stdout.toString() : "";
const stdErr = error?.stderr ? error.stderr.toString() : "";
lastOutput = [stdOut, stdErr, error?.message]
.filter(Boolean)
.join("\n");
console.error("Dependency check command failed:", lastOutput);
}
const elapsed = Date.now() - startTime;
const remaining = Math.max(0, maxDurationMs - elapsed);
const waitMs = Math.min(retryIntervalMs, remaining);
if (waitMs <= 0) break;
console.log(`Waiting ${waitMs}ms before retry...`);
await new Promise((resolve) => setTimeout(resolve, waitMs));
}
throw new Error(
`Dependencies not fully installed in ${appPath} after 3 minutes. Last output: ${lastOutput}`,
);
}
async setUpDyadProvider() {
await this.page
.locator("div")
.filter({ hasText: /^DyadNeeds Setup$/ })
.nth(1)
.click();
await this.page.getByRole("textbox", { name: "Set Dyad API Key" }).click();
await this.page
.getByRole("textbox", { name: "Set Dyad API Key" })
.fill("testdyadkey");
await this.page.getByRole("button", { name: "Save Key" }).click();
}
async importApp(appDir: string) {
await this.page.getByRole("button", { name: "Import App" }).click();
await eph.stubDialog(this.electronApp, "showOpenDialog", {
filePaths: [path.join(__dirname, "..", "fixtures", "import-app", appDir)],
});
await this.page.getByRole("button", { name: "Select Folder" }).click();
await this.page.getByRole("button", { name: "Import" }).click();
}
async selectChatMode(
mode: "build" | "ask" | "agent" | "local-agent" | "basic-agent" | "plan",
) {
await this.page.getByTestId("chat-mode-selector").click();
const mapping: Record<string, string> = {
build: "Build Generate and edit code",
ask: "Ask Ask",
agent: "Build with MCP",
"local-agent": "Agent v2",
"basic-agent": "Basic Agent", // For free users
plan: "Plan.*Design before you build",
};
const optionName = mapping[mode];
await this.page
.getByRole("option", {
name: new RegExp(optionName),
})
.click();
}
async selectLocalAgentMode() {
await this.selectChatMode("local-agent");
}
async openContextFilesPicker() {
// Programmatically dismiss toasts using the sonner API by clicking any visible close buttons
const toastCloseButtons = this.page.locator(
"[data-sonner-toast] button[data-close-button]",
);
const closeCount = await toastCloseButtons.count();
for (let i = 0; i < closeCount; i++) {
await toastCloseButtons
.nth(i)
.click()
.catch(() => {});
}
// If close buttons don't work, click outside to dismiss
if ((await this.page.locator("[data-sonner-toast]").count()) > 0) {
// Click somewhere safe to dismiss toasts
await this.page.mouse.click(10, 10);
await this.page.waitForTimeout(300);
}
// Open the auxiliary actions menu
await this.getChatInputContainer()
.getByTestId("auxiliary-actions-menu")
.click();
// Click on "Codebase context" to open the popover
await this.page.getByTestId("codebase-context-trigger").click();
// Wait for the popover content to be visible
await this.page
.getByTestId("manual-context-files-input")
.waitFor({ state: "visible" });
return new ContextFilesPickerDialog(this.page, async () => {
// Close the popover first
await this.page.keyboard.press("Escape");
// Wait a bit for the popover to close, then close the dropdown menu
await this.page
.getByTestId("manual-context-files-input")
.waitFor({ state: "hidden" });
await this.page.keyboard.press("Escape");
});
}
async openProModesDialog({
location = "chat-input-container",
}: {
location?: "chat-input-container" | "home-chat-input-container";
} = {}): Promise<ProModesDialog> {
const proButton = this.page
// Assumes you're on the chat page.
.getByTestId(location)
.getByRole("button", { name: "Pro", exact: true });
await proButton.click();
return new ProModesDialog(this.page, async () => {
await proButton.click();
});
}
async snapshotDialog() {
await expect(this.page.getByRole("dialog")).toMatchAriaSnapshot();
}
async snapshotAppFiles({ name, files }: { name: string; files?: string[] }) {
const currentAppName = await this.getCurrentAppName();
if (!currentAppName) {
throw new Error("No app selected");
}
const normalizedAppName = currentAppName.toLowerCase().replace(/-/g, "");
const appPath = await this.getCurrentAppPath();
if (!appPath || !fs.existsSync(appPath)) {
throw new Error(`App path does not exist: ${appPath}`);
}
await expect(() => {
let filesData = generateAppFilesSnapshotData(appPath, appPath);
// Sort by relative path to ensure deterministic output
filesData.sort((a, b) => a.relativePath.localeCompare(b.relativePath));
if (files) {
filesData = filesData.filter((file) =>
files.some(
(f) => normalizePath(f) === normalizePath(file.relativePath),
),
);
}
const snapshotContent = filesData
.map(
(file) =>
`=== ${file.relativePath.replace(normalizedAppName, "[[normalizedAppName]]")} ===\n${file.content
.split(normalizedAppName)
.join("[[normalizedAppName]]")
.split(currentAppName)
.join("[[appName]]")}`,
)
.join("\n\n");
if (name) {
expect(snapshotContent).toMatchSnapshot(name + ".txt");
} else {
expect(snapshotContent).toMatchSnapshot();
}
}).toPass();
}
async snapshotMessages({
replaceDumpPath = false,
timeout,
}: { replaceDumpPath?: boolean; timeout?: number } = {}) {
// NOTE: once you have called this, you can NOT manipulate the UI anymore or React will break.
if (replaceDumpPath) {
await this.page.evaluate(() => {
const messagesList = document.querySelector(
"[data-testid=messages-list]",
);
if (!messagesList) {
throw new Error("Messages list not found");
}
// Scrub compaction backup paths embedded in message text
// e.g. .dyad/chats/1/compaction-2026-02-05T21-25-24-285Z.md
messagesList.innerHTML = messagesList.innerHTML.replace(
/\.dyad\/chats\/\d+\/compaction-[^\s<"]+\.md/g,
"[[compaction-backup-path]]",
);
messagesList.innerHTML = messagesList.innerHTML.replace(
/\[\[dyad-dump-path=([^\]]+)\]\]/g,
"[[dyad-dump-path=*]]",
);
});
}
await expect(this.page.getByTestId("messages-list")).toMatchAriaSnapshot({
timeout,
});
}
async approveProposal() {
await this.page.getByTestId("approve-proposal-button").click();
}
async rejectProposal() {
await this.page.getByTestId("reject-proposal-button").click();
}
async clickRestart() {
await this.page.getByRole("button", { name: "Restart" }).click();
}
////////////////////////////////
// Inline code editor
////////////////////////////////
async clickEditButton() {
await this.page.locator('button:has-text("Edit")').first().click();
}
async editFileContent(content: string) {
const editor = this.page.locator(".monaco-editor textarea").first();
await editor.focus();
await editor.press("Home");
await editor.type(content);
}
async saveFile() {
await this.page.locator('[data-testid="save-file-button"]').click();
}
async cancelEdit() {
await this.page.locator('button:has-text("Cancel")').first().click();
}
////////////////////////////////
// Preview panel
////////////////////////////////
async selectPreviewMode(
mode:
| "code"
| "problems"
| "preview"
| "configure"
| "security"
| "publish",
) {
await this.page.getByTestId(`${mode}-mode-button`).click();
}
async clickChatActivityButton() {
await this.page.getByTestId("chat-activity-button").click();
}
async snapshotChatActivityList() {
await expect(
this.page.getByTestId("chat-activity-list"),
).toMatchAriaSnapshot();
}
async clickRecheckProblems() {
await this.page.getByTestId("recheck-button").click();
}
async clickFixAllProblems() {
await this.page.getByTestId("fix-all-button").click();
await this.waitForChatCompletion();
}
async snapshotProblemsPane() {
await expect(this.page.getByTestId("problems-pane")).toMatchAriaSnapshot({
timeout: Timeout.MEDIUM,
});
}
async clickRebuild() {
await this.clickPreviewMoreOptions();
await this.page.getByText("Rebuild").click();
}
async clickTogglePreviewPanel() {
await this.page.getByTestId("toggle-preview-panel-button").click();
}
async clickPreviewPickElement() {
await this.page
.getByTestId("preview-pick-element-button")
.click({ timeout: Timeout.EXTRA_LONG });
}
async clickDeselectComponent(options?: { index?: number }) {
const buttons = this.page.getByRole("button", {
name: "Deselect component",
});
if (options?.index !== undefined) {
await buttons.nth(options.index).click();
} else {
await buttons.first().click();
}
}
async clickPreviewMoreOptions() {
await this.page.getByTestId("preview-more-options-button").click();
}
async clickPreviewRefresh() {
await this.page.getByTestId("preview-refresh-button").click();
}
async clickPreviewNavigateBack() {
await this.page.getByTestId("preview-navigate-back-button").click();
}
async clickPreviewNavigateForward() {
await this.page.getByTestId("preview-navigate-forward-button").click();
}
async clickPreviewOpenBrowser() {
await this.page.getByTestId("preview-open-browser-button").click();
}
async clickPreviewAnnotatorButton() {
await this.page
.getByTestId("preview-annotator-button")
.click({ timeout: Timeout.EXTRA_LONG });
}
async waitForAnnotatorMode() {
// Wait for the annotator toolbar to be visible
await expect(this.page.getByRole("button", { name: "Select" })).toBeVisible(
{
timeout: Timeout.MEDIUM,
},
);
}
async clickAnnotatorSubmit() {
await this.page.getByRole("button", { name: "Add to Chat" }).click();
}
locateLoadingAppPreview() {
return this.page.getByText("Preparing app preview...");
}
locateStartingAppPreview() {
return this.page.getByText("Starting your app server...");
}
getPreviewIframeElement() {
return this.page.getByTestId("preview-iframe-element");
}
expectPreviewIframeIsVisible() {
return expect(this.getPreviewIframeElement()).toBeVisible({
timeout: Timeout.LONG,
});
}
async clickFixErrorWithAI() {
await this.page.getByRole("button", { name: "Fix error with AI" }).click();
}
async clickCopyErrorMessage() {
await this.page
.getByTestId("preview-error-banner")
.getByRole("button", { name: /Copy/ })
.click();
}
async getClipboardText(): Promise<string> {
return await this.page.evaluate(() => navigator.clipboard.readText());
}
async clickFixAllErrors() {
await this.page.getByRole("button", { name: /Fix All Errors/ }).click();
}
async snapshotPreviewErrorBanner() {
await expect(this.locatePreviewErrorBanner()).toMatchAriaSnapshot({
timeout: Timeout.LONG,
});
}
locatePreviewErrorBanner() {
return this.page.getByTestId("preview-error-banner");
}
async snapshotChatInputContainer() {
await expect(this.getChatInputContainer()).toMatchAriaSnapshot();
}
getSelectedComponentsDisplay() {
return this.page.getByTestId("selected-component-display");
}
async snapshotSelectedComponentsDisplay() {
await expect(this.getSelectedComponentsDisplay()).toMatchAriaSnapshot();
}
async snapshotPreview({ name }: { name?: string } = {}) {
const iframe = this.getPreviewIframeElement();
await expect(iframe.contentFrame().locator("body")).toMatchAriaSnapshot({
name,
timeout: Timeout.LONG,
});
}
////////////////////////////////
// Security review
////////////////////////////////
async clickRunSecurityReview() {
const runSecurityReviewButton = this.page
.getByRole("button", { name: "Run Security Review" })
.first();
await runSecurityReviewButton.click();
await runSecurityReviewButton.waitFor({ state: "hidden" });
await this.waitForChatCompletion();
}
async snapshotSecurityFindingsTable() {
await expect(
this.page.getByTestId("security-findings-table"),
).toMatchAriaSnapshot();
}
async snapshotServerDump(
type: "all-messages" | "last-message" | "request" = "all-messages",
{ name = "", dumpIndex = -1 }: { name?: string; dumpIndex?: number } = {},
) {
await this.waitForChatCompletion();
// Get the text content of the messages list
const messagesListText = await this.page
.getByTestId("messages-list")
.textContent();
// Find ALL dump paths using global regex
const dumpPathMatches = messagesListText?.match(
/\[\[dyad-dump-path=([^\]]+)\]\]/g,
);
if (!dumpPathMatches || dumpPathMatches.length === 0) {
throw new Error("No dump path found in messages list");
}
// Extract the actual paths from the matches
const dumpPaths = dumpPathMatches
.map((match) => {
const pathMatch = match.match(/\[\[dyad-dump-path=([^\]]+)\]\]/);
return pathMatch ? pathMatch[1] : null;
})
.filter(Boolean);
// Select the dump path based on index
// -1 means last, -2 means second to last, etc.
// 0 means first, 1 means second, etc.
const selectedIndex =
dumpIndex < 0 ? dumpPaths.length + dumpIndex : dumpIndex;
if (selectedIndex < 0 || selectedIndex >= dumpPaths.length) {
throw new Error(
`Dump index ${dumpIndex} is out of range. Found ${dumpPaths.length} dump paths.`,
);
}
const dumpFilePath = dumpPaths[selectedIndex];
if (!dumpFilePath) {
throw new Error("No dump file path found");
}
// Read the JSON file
const dumpContent: string = (fs.readFileSync(dumpFilePath, "utf-8") as any)
.replaceAll(/\[\[dyad-dump-path=([^\]]+)\]\]/g, "[[dyad-dump-path=*]]")
// Stabilize compaction backup file paths embedded in message text
// e.g. .dyad/chats/1/compaction-2026-02-05T21-25-24-285Z.md
.replaceAll(
/\.dyad\/chats\/\d+\/compaction-[^\s"\\]+\.md/g,
"[[compaction-backup-path]]",
);
// Perform snapshot comparison
const parsedDump = JSON.parse(dumpContent);
if (type === "request") {
if (parsedDump["body"]["input"]) {
parsedDump["body"]["input"] = parsedDump["body"]["input"].map(
(input: any) => {
if (input.role === "system") {
input.content = "[[SYSTEM_MESSAGE]]";
}
return input;
},
);
}
if (parsedDump["body"]["messages"]) {
parsedDump["body"]["messages"] = parsedDump["body"]["messages"].map(
(message: any) => {
if (message.role === "system") {
message.content = "[[SYSTEM_MESSAGE]]";
}
return message;
},
);
}
// Normalize fileIds to be deterministic based on content
normalizeVersionedFiles(parsedDump);
// Normalize item_reference IDs (e.g., msg_1234567890) to be deterministic
normalizeItemReferences(parsedDump);
// Normalize tool_call IDs (e.g., call_1234567890_0) to be deterministic
normalizeToolCallIds(parsedDump);
expect(
JSON.stringify(parsedDump, null, 2).replace(/\\r\\n/g, "\\n"),
).toMatchSnapshot(name);
return;
}
expect(
prettifyDump(
// responses API
parsedDump["body"]["input"] ??
// chat completion API
parsedDump["body"]["messages"],
{
onlyLastMessage: type === "last-message",
},
),
).toMatchSnapshot(name);
}
async waitForChatCompletion() {
await expect(this.getRetryButton()).toBeVisible({
timeout: Timeout.MEDIUM,
});
}
async clickRetry() {
await this.getRetryButton().click();
}
async clickUndo() {
await this.getUndoButton().click();
}
private getRetryButton() {
return this.page.getByRole("button", { name: "Retry" });
}
private getUndoButton() {
return this.page.getByRole("button", { name: "Undo" });
}
getHomeChatInputContainer() {
return this.page.getByTestId("home-chat-input-container");
}
getChatInputContainer() {
return this.page.getByTestId("chat-input-container");
}
getChatInput() {
return this.page.locator(
'[data-lexical-editor="true"][aria-placeholder^="Ask Dyad to build"]',
);
}
/**
* Clears the Lexical chat input using keyboard shortcuts (Meta+A, Backspace).
* Uses toPass() for resilience since Lexical may need time to update its state.
*/
async clearChatInput() {
const chatInput = this.getChatInput();
await chatInput.click();
await this.page.keyboard.press("ControlOrMeta+a");
await this.page.keyboard.press("Backspace");
await expect(async () => {
const text = await chatInput.textContent();
expect(text?.trim()).toBe("");
}).toPass({ timeout: Timeout.SHORT });
}
/**
* Opens the chat history menu by clearing the input and pressing ArrowUp.
* Uses toPass() for resilience since the Lexical editor may need time to
* update its state before the history menu can be triggered.
*/
async openChatHistoryMenu() {
const historyMenu = this.page.locator('[data-mentions-menu="true"]');
await expect(async () => {
await this.clearChatInput();
await this.page.keyboard.press("ArrowUp");
await expect(historyMenu).toBeVisible({ timeout: 500 });
}).toPass({ timeout: Timeout.SHORT });
}
clickNewChat({ index = 0 }: { index?: number } = {}) {
// There is two new chat buttons...
return this.page
.getByRole("button", { name: "New Chat" })
.nth(index)
.click();
}
async clickBackButton() {
await this.page.getByRole("button", { name: "Back" }).click();
}
async toggleTokenBar() {
// Need to make sure it's NOT visible yet to avoid a race when we opened
// the auxiliary actions menu earlier.
await expect(this.page.getByTestId("token-bar-toggle")).not.toBeVisible();
await this.getChatInputContainer()
.getByTestId("auxiliary-actions-menu")
.click();
await this.page.getByTestId("token-bar-toggle").click();
}
async sendPrompt(
prompt: string,
{ skipWaitForCompletion = false }: { skipWaitForCompletion?: boolean } = {},
) {
await this.getChatInput().click();
await this.getChatInput().fill(prompt);
await this.page.getByRole("button", { name: "Send message" }).click();
if (!skipWaitForCompletion) {
await this.waitForChatCompletion();
}
}
async selectModel({ provider, model }: { provider: string; model: string }) {
await this.page.getByTestId("model-picker").click();
await this.page.getByText(provider, { exact: true }).click();
await this.page.getByText(model, { exact: true }).click();
}
async selectTestModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("test-provider").click();
await this.page.getByText("test-model").click();
}
async selectTestOllamaModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Local models").click();
await this.page.getByText("Ollama", { exact: true }).click();
await this.page.getByText("Testollama", { exact: true }).click();
}
async selectTestLMStudioModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Local models").click();
await this.page.getByText("LM Studio", { exact: true }).click();
// Both of the elements that match "lmstudio-model-1" are the same button, so we just pick the first.
await this.page
.getByText("lmstudio-model-1", { exact: true })
.first()
.click();
}
async selectTestAzureModel() {
await this.page.getByTestId("model-picker").click();
await this.page.getByText("Other AI providers").click();
await this.page.getByText("Azure OpenAI", { exact: true }).click();
await this.page.getByText("GPT-5", { exact: true }).click();
}
async setUpAzure({ autoApprove = false }: { autoApprove?: boolean } = {}) {
await this.githubConnector.clearPushEvents();
await this.goToSettingsTab();
if (autoApprove) {
await this.toggleAutoApprove();
}
// Azure should already be configured via environment variables
// so we don't need additional setup steps like setUpDyadProvider
await this.goToAppsTab();
}
async setUpTestProvider() {
await this.page.getByText("Add custom providerConnect to").click();
// Fill out provider dialog
await this.page
.getByRole("textbox", { name: "Provider ID" })
.fill("testing");
await this.page.getByRole("textbox", { name: "Display Name" }).click();
await this.page
.getByRole("textbox", { name: "Display Name" })
.fill("test-provider");
await this.page.getByText("API Base URLThe base URL for").click();
await this.page
.getByRole("textbox", { name: "API Base URL" })
.fill("http://localhost:3500/v1");
await this.page.getByRole("button", { name: "Add Provider" }).click();
}
async setUpTestModel() {
await this.page.getByRole("heading", { name: "test-provider" }).click();
await this.page.getByRole("button", { name: "Add Custom Model" }).click();
await this.page
.getByRole("textbox", { name: "Model ID*" })
.fill("test-model");
await this.page.getByRole("textbox", { name: "Model ID*" }).press("Tab");
await this.page.getByRole("textbox", { name: "Name*" }).fill("test-model");
await this.page.getByRole("button", { name: "Add Model" }).click();
}
async addCustomTestModel({
name,
contextWindow,
}: {
name: string;
contextWindow?: number;
}) {
await this.page.getByRole("heading", { name: "test-provider" }).click();
await this.page.getByRole("button", { name: "Add Custom Model" }).click();
await this.page.getByRole("textbox", { name: "Model ID*" }).fill(name);
await this.page.getByRole("textbox", { name: "Model ID*" }).press("Tab");
await this.page.getByRole("textbox", { name: "Name*" }).fill(name);
if (contextWindow) {
await this.page.locator("#context-window").fill(String(contextWindow));
}
await this.page.getByRole("button", { name: "Add Model" }).click();
}
async setUpTestProviderApiKey() {
// Fill in a test API key for the custom provider
await this.page
.getByPlaceholder(/Enter new.*API Key here/)
.fill("test-api-key-12345");
await this.page.getByRole("button", { name: "Save Key" }).click();
// Wait for the key to be saved
await expect(this.page.getByText("test-api-key-12345")).toBeVisible();
}
async goToSettingsTab() {
await this.page.getByRole("link", { name: "Settings" }).click();
}
async goToLibraryTab() {
await this.page.getByRole("link", { name: "Library" }).click();
}
async createPrompt({
title,
description,
content,
}: {
title: string;
description?: string;
content: string;
}) {
await this.page.getByRole("button", { name: "New Prompt" }).click();
await this.page.getByRole("textbox", { name: "Title" }).fill(title);
if (description) {
await this.page
.getByRole("textbox", { name: "Description (optional)" })
.fill(description);
}
await this.page.getByRole("textbox", { name: "Content" }).fill(content);
await this.page.getByRole("button", { name: "Save" }).click();
}
getTitleBarAppNameButton() {
return this.page.getByTestId("title-bar-app-name-button");
}
getAppListItem({ appName }: { appName: string }) {
return this.page.getByTestId(`app-list-item-${appName}`);
}
async isCurrentAppNameNone() {
await expect(async () => {
await expect(this.getTitleBarAppNameButton()).toContainText(
"no app selected",
);
}).toPass();
}
async getCurrentAppName() {
// Make sure to wait for the app to be set to avoid a race condition.
await expect(async () => {
await expect(this.getTitleBarAppNameButton()).not.toContainText(
"no app selected",
);
}).toPass();
return (await this.getTitleBarAppNameButton().textContent())?.replace(
"App: ",
"",
);
}
async getCurrentAppPath() {
const currentAppName = await this.getCurrentAppName();
if (!currentAppName) {
throw new Error("No current app name found");
}
return this.getAppPath({ appName: currentAppName });
}
async configureGitUser({
email = "test@example.com",
name = "Test User",
disableGpgSign = true,
}: {
email?: string;
name?: string;
disableGpgSign?: boolean;
} = {}) {
const appPath = await this.getCurrentAppPath();
if (!appPath) {
throw new Error("App path not found");
}
execSync(`git config user.email '${email}'`, { cwd: appPath });
execSync(`git config user.name '${name}'`, { cwd: appPath });
if (disableGpgSign) {
execSync("git config commit.gpgsign false", { cwd: appPath });
}
}
getAppPath({ appName }: { appName: string }) {
return path.join(this.userDataDir, "dyad-apps", appName);
}
async clickAppListItem({ appName }: { appName: string }) {
await this.page.getByTestId(`app-list-item-${appName}`).click();
}
async clickOpenInChatButton() {
await this.page.getByRole("button", { name: "Open in Chat" }).click();
}
locateAppUpgradeButton({ upgradeId }: { upgradeId: string }) {
return this.page.getByTestId(`app-upgrade-${upgradeId}`);
}
async clickAppUpgradeButton({ upgradeId }: { upgradeId: string }) {
await this.locateAppUpgradeButton({ upgradeId }).click();
}
async expectAppUpgradeButtonIsNotVisible({
upgradeId,
}: {
upgradeId: string;
}) {
await expect(this.locateAppUpgradeButton({ upgradeId })).toBeHidden({
timeout: Timeout.MEDIUM,
});
}
async expectNoAppUpgrades() {
await expect(this.page.getByTestId("no-app-upgrades-needed")).toBeVisible({
timeout: Timeout.LONG,
});
}
async clickAppDetailsRenameAppButton() {
await this.page.getByTestId("app-details-rename-app-button").click();
}
async clickAppDetailsMoreOptions() {
await this.page.getByTestId("app-details-more-options-button").click();
}
async clickAppDetailsCopyAppButton() {
await this.page.getByRole("button", { name: "Copy app" }).click();
}
async clickConnectSupabaseButton() {
await this.page.getByTestId("connect-supabase-button").click();
}
////////////////////////////////
// Settings related
////////////////////////////////
async toggleAutoApprove() {
await this.page.getByRole("switch", { name: "Auto-approve" }).click();
}
async toggleLocalAgentMode() {
await this.page.getByRole("switch", { name: "Enable Agent v2" }).click();
}
async toggleNativeGit() {
await this.page.getByRole("switch", { name: "Enable Native Git" }).click();
}
async toggleAutoFixProblems() {
await this.page.getByRole("switch", { name: "Auto-fix problems" }).click();
}
/**
* Records the current settings state for later comparison.
* Use with `snapshotSettingsDelta()` to snapshot only what changed.
*/
recordSettings(): Record<string, unknown> {
const settingsPath = path.join(this.userDataDir, "user-settings.json");
const settingsContent = fs.readFileSync(settingsPath, "utf-8");
return JSON.parse(settingsContent);
}
/**
* Snapshots only the differences between the current settings and a previously recorded state.
* Output is in git diff style for easy reading.
*/
snapshotSettingsDelta(beforeSettings: Record<string, unknown>) {
const afterSettings = this.recordSettings();
const diffLines: string[] = [];
const allKeys = new Set([
...Object.keys(beforeSettings),
...Object.keys(afterSettings),
]);
// Sort keys for deterministic output
const sortedKeys = Array.from(allKeys).sort();
// Keys whose values should be redacted for deterministic snapshots
const redactedKeys: Record<string, string> = {
telemetryUserId: "[UUID]",
lastShownReleaseNotesVersion: "[scrubbed]",
};
for (const key of sortedKeys) {
const beforeValue = beforeSettings[key];
const afterValue = afterSettings[key];
const beforeExists = key in beforeSettings;
const afterExists = key in afterSettings;
// Format value with diff marker on each line for multiline values
// Redact certain keys for deterministic snapshots
const formatValue = (val: unknown, marker: "+" | "-") => {
const displayVal = key in redactedKeys ? redactedKeys[key] : val;
const lines = JSON.stringify(displayVal, null, 2).split("\n");
return lines
.map((line, i) => (i === 0 ? line : `${marker} ${line}`))
.join("\n");
};
if (!beforeExists && afterExists) {
// Added
diffLines.push(`+ "${key}": ${formatValue(afterValue, "+")}`);
} else if (beforeExists && !afterExists) {
// Removed
diffLines.push(`- "${key}": ${formatValue(beforeValue, "-")}`);
} else if (JSON.stringify(beforeValue) !== JSON.stringify(afterValue)) {
// Changed
diffLines.push(`- "${key}": ${formatValue(beforeValue, "-")}`);
diffLines.push(`+ "${key}": ${formatValue(afterValue, "+")}`);
}
}
expect(diffLines.join("\n")).toMatchSnapshot();
}
async toggleAutoUpdate() {
await this.page.getByRole("switch", { name: "Auto-update" }).click();
}
async changeReleaseChannel(channel: "stable" | "beta") {
// await page.getByRole('combobox').filter({ hasText: 'Stable' }).click();
// await page.getByRole('option', { name: 'Beta' }).dblclick();
await this.page.getByRole("combobox", { name: "Release Channel" }).click();
await this.page
.getByRole("option", { name: channel === "stable" ? "Stable" : "Beta" })
.click();
}
async clickTelemetryAccept() {
await this.page.getByTestId("telemetry-accept-button").click();
}
async clickTelemetryReject() {
await this.page.getByTestId("telemetry-reject-button").click();
}
async clickTelemetryLater() {
await this.page.getByTestId("telemetry-later-button").click();
}
async goToAppsTab() {
await this.page.getByRole("link", { name: "Apps" }).click();
await expect(this.page.getByText("Build a new app")).toBeVisible();
}
async goToChatTab() {
await this.page.getByRole("link", { name: "Chat" }).click();
}
async goToHubTab() {
await this.page.getByRole("link", { name: "Hub" }).click();
}
async selectTemplate(templateName: string) {
await this.page.getByRole("img", { name: templateName }).click();
}
async goToHubAndSelectTemplate(templateName: "Next.js Template") {
await this.goToHubTab();
await this.selectTemplate(templateName);
await this.goToAppsTab();
}
////////////////////////////////
// Toast assertions
////////////////////////////////
async expectNoToast() {
await expect(this.page.locator("[data-sonner-toast]")).toHaveCount(0);
}
async waitForToast(
type?: "success" | "error" | "warning" | "info",
timeout = 5000,
) {
const selector = type
? `[data-sonner-toast][data-type="${type}"]`
: "[data-sonner-toast]";
await this.page.waitForSelector(selector, { timeout });
}
async waitForToastWithText(text: string, timeout = Timeout.MEDIUM) {
await this.page.waitForSelector(`[data-sonner-toast]:has-text("${text}")`, {
timeout,
});
}
async assertToastVisible(type?: "success" | "error" | "warning" | "info") {
const selector = type
? `[data-sonner-toast][data-type="${type}"]`
: "[data-sonner-toast]";
await expect(this.page.locator(selector)).toBeVisible();
}
async assertToastWithText(text: string) {
await expect(
this.page.locator(`[data-sonner-toast]:has-text("${text}")`),
).toBeVisible();
}
async dismissAllToasts() {
// Click all close buttons if they exist
const closeButtons = this.page.locator(
"[data-sonner-toast] button[data-close-button]",
);
const count = await closeButtons.count();
for (let i = 0; i < count; i++) {
await closeButtons.nth(i).click();
}
}
async sleep(ms: number) {
await new Promise((resolve) => setTimeout(resolve, ms));
}
////////////////////////////////
// Agent Tool Consent Banner
////////////////////////////////
getAgentConsentBanner() {
return this.page
.getByRole("button", { name: "Always allow" })
.locator("..");
}
async waitForAgentConsentBanner(timeout = Timeout.MEDIUM) {
await expect(
this.page.getByRole("button", { name: "Always allow" }),
).toBeVisible({ timeout });
}
async clickAgentConsentAlwaysAllow() {
await this.page.getByRole("button", { name: "Always allow" }).click();
}
async clickAgentConsentAllowOnce() {
await this.page.getByRole("button", { name: "Allow once" }).click();
}
async clickAgentConsentDecline() {
await this.page.getByRole("button", { name: "Decline" }).click();
}
////////////////////////////////
// Test-only: Node.js Mock Control
////////////////////////////////
/**
* Set the mock state for Node.js installation status.
* @param installed - true = mock as installed, false = mock as not installed, null = use real check
*/
async setNodeMock(installed: boolean | null) {
await this.page.evaluate(async (installed) => {
await (window as any).electron.ipcRenderer.invoke("test:set-node-mock", {
installed,
});
}, installed);
}
}
interface ElectronConfig {
preLaunchHook?: ({ userDataDir }: { userDataDir: string }) => Promise<void>;
showSetupScreen?: boolean;
}
// From https://github.com/microsoft/playwright/issues/8208#issuecomment-1435475930
//
// Note how we mark the fixture as { auto: true }.
// This way it is always instantiated, even if the test does not use it explicitly.
export const test = base.extend<{
electronConfig: ElectronConfig;
attachScreenshotsToReport: void;
electronApp: ElectronApplication;
po: PageObject;
}>({
electronConfig: [
async ({}, use) => {
// Default configuration - tests can override this fixture
await use({});
},
{ auto: true },
],
po: [
async ({ electronApp }, use) => {
const page = await electronApp.firstWindow();
const po = new PageObject(electronApp, page, {
userDataDir: (electronApp as any).$dyadUserDataDir,
});
await use(po);
},
{ auto: true },
],
attachScreenshotsToReport: [
async ({ electronApp }, use, testInfo) => {
await use();
// After the test we can check whether the test passed or failed.
if (testInfo.status !== testInfo.expectedStatus) {
const page = await electronApp.firstWindow();
try {
const screenshot = await page.screenshot({ timeout: 5_000 });
await testInfo.attach("screenshot", {
body: screenshot,
contentType: "image/png",
});
} catch (error) {
console.error("Error taking screenshot on failure", error);
}
}
},
{ auto: true },
],
electronApp: [
async ({ electronConfig }, use) => {
// find the latest build in the out directory
const latestBuild = eph.findLatestBuild();
// parse the directory and find paths and other info
const appInfo = eph.parseElectronApp(latestBuild);
process.env.OLLAMA_HOST = "http://localhost:3500/ollama";
process.env.LM_STUDIO_BASE_URL_FOR_TESTING =
"http://localhost:3500/lmstudio";
process.env.DYAD_ENGINE_URL = "http://localhost:3500/engine/v1";
process.env.DYAD_GATEWAY_URL = "http://localhost:3500/gateway/v1";
process.env.E2E_TEST_BUILD = "true";
if (!electronConfig.showSetupScreen) {
// This is just a hack to avoid the AI setup screen.
process.env.OPENAI_API_KEY = "sk-test";
}
const baseTmpDir = os.tmpdir();
const userDataDir = path.join(baseTmpDir, `dyad-e2e-tests-${Date.now()}`);
if (electronConfig.preLaunchHook) {
await electronConfig.preLaunchHook({ userDataDir });
}
const electronApp = await electron.launch({
args: [
appInfo.main,
"--enable-logging",
`--user-data-dir=${userDataDir}`,
],
executablePath: appInfo.executable,
// Strong suspicion this is causing issues on Windows with tests hanging due to error:
// ffmpeg failed to write: Error [ERR_STREAM_WRITE_AFTER_END]: write after end
// recordVideo: {
// dir: "test-results",
// },
});
(electronApp as any).$dyadUserDataDir = userDataDir;
console.log("electronApp launched!");
if (showDebugLogs) {
// Listen to main process output immediately
electronApp.process().stdout?.on("data", (data) => {
console.log(`MAIN_PROCESS_STDOUT: ${data.toString()}`);
});
electronApp.process().stderr?.on("data", (data) => {
console.error(`MAIN_PROCESS_STDERR: ${data.toString()}`);
});
}
electronApp.on("close", () => {
console.log(`Electron app closed listener:`);
});
electronApp.on("window", async (page) => {
const filename = page.url()?.split("/").pop();
console.log(`Window opened: ${filename}`);
// capture errors
page.on("pageerror", (error) => {
console.error(error);
});
// capture console messages
page.on("console", (msg) => {
console.log(msg.text());
});
});
await use(electronApp);
// Why are we doing a force kill on Windows?
//
// Otherwise, Playwright will just hang on the test cleanup
// because the electron app does NOT ever fully quit due to
// Windows' strict resource locking (e.g. file locking).
if (os.platform() === "win32") {
try {
console.log("[cleanup:start] Killing dyad.exe");
console.time("taskkill");
execSync("taskkill /f /t /im dyad.exe");
console.timeEnd("taskkill");
console.log("[cleanup:end] Killed dyad.exe");
} catch (error) {
console.warn(
"Failed to kill dyad.exe: (continuing with test cleanup)",
error,
);
}
} else {
await electronApp.close();
}
},
{ auto: true },
],
});
export function testWithConfig(config: ElectronConfig) {
return test.extend({
electronConfig: async ({}, use) => {
await use(config);
},
});
}
export function testWithConfigSkipIfWindows(config: ElectronConfig) {
if (os.platform() === "win32") {
return test.skip;
}
return test.extend({
electronConfig: async ({}, use) => {
await use(config);
},
});
}
// Wrapper that skips tests on Windows platform
export const testSkipIfWindows = os.platform() === "win32" ? test.skip : test;
function prettifyDump(
allMessages: {
role: string;
content: string | Array<{}>;
}[],
{ onlyLastMessage = false }: { onlyLastMessage?: boolean } = {},
) {
const messages = onlyLastMessage ? allMessages.slice(-1) : allMessages;
return messages
.map((message) => {
const content = Array.isArray(message.content)
? JSON.stringify(message.content)
: message.content
.replace(BUILD_SYSTEM_PREFIX, "\n${BUILD_SYSTEM_PREFIX}")
.replace(BUILD_SYSTEM_POSTFIX, "${BUILD_SYSTEM_POSTFIX}")
// Normalize line endings to always use \n
.replace(/\r\n/g, "\n")
// We remove package.json because it's flaky.
// Depending on whether pnpm install is run, it will be modified,
// and the contents and timestamp (thus affecting order) will be affected.
.replace(
/\n<dyad-file path="package\.json">[\s\S]*?<\/dyad-file>\n/g,
"",
);
return `===\nrole: ${message.role}\nmessage: ${content}`;
})
.join("\n\n");
}
function normalizePath(path: string): string { // Re-export constants
return path.replace(/\\/g, "/"); export { Timeout, showDebugLogs } from "./constants";
}
// Re-export fixtures and test utilities
export {
test,
testWithConfig,
testWithConfigSkipIfWindows,
testSkipIfWindows,
type ElectronConfig,
} from "./fixtures";
// Re-export page objects
export {
PageObject,
ContextFilesPickerDialog,
ProModesDialog,
GitHubConnector,
ChatActions,
PreviewPanel,
CodeEditor,
SecurityReview,
ToastNotifications,
AgentConsent,
Navigation,
ModelPicker,
Settings,
AppManagement,
PromptLibrary,
} from "./page-objects";
// Re-export utilities (for tests that may need direct access)
export {
normalizeItemReferences,
normalizeToolCallIds,
normalizeVersionedFiles,
normalizePath,
prettifyDump,
} from "./utils";
/**
* Utility for prettifying server dump data for snapshot comparisons.
*/
import {
BUILD_SYSTEM_POSTFIX,
BUILD_SYSTEM_PREFIX,
} from "@/prompts/system_prompt";
export interface PrettifyDumpOptions {
onlyLastMessage?: boolean;
}
/**
* Prettifies a dump of messages for snapshot comparison.
* Normalizes line endings, removes flaky content like package.json,
* and formats the output for readability.
*/
export function prettifyDump(
allMessages: {
role: string;
content: string | Array<{}>;
}[],
{ onlyLastMessage = false }: PrettifyDumpOptions = {},
): string {
const messages = onlyLastMessage ? allMessages.slice(-1) : allMessages;
return messages
.map((message) => {
const content = Array.isArray(message.content)
? JSON.stringify(message.content)
: message.content
.replace(BUILD_SYSTEM_PREFIX, "\n${BUILD_SYSTEM_PREFIX}")
.replace(BUILD_SYSTEM_POSTFIX, "${BUILD_SYSTEM_POSTFIX}")
// Normalize line endings to always use \n
.replace(/\r\n/g, "\n")
// We remove package.json because it's flaky.
// Depending on whether pnpm install is run, it will be modified,
// and the contents and timestamp (thus affecting order) will be affected.
.replace(
/\n<dyad-file path="package\.json">[\s\S]*?<\/dyad-file>\n/g,
"",
);
return `===\nrole: ${message.role}\nmessage: ${content}`;
})
.join("\n\n");
}
/**
* Barrel file for utility exports.
*/
export {
normalizeItemReferences,
normalizeToolCallIds,
normalizeVersionedFiles,
normalizePath,
} from "./normalization";
export { prettifyDump, type PrettifyDumpOptions } from "./dump-prettifier";
/**
* Utility functions for normalizing test data to ensure deterministic snapshots.
*/
/**
* Normalizes item_reference IDs in the input array to be deterministic.
* item_reference objects have the shape { type: "item_reference", id: "msg_..." }
* where the ID is a timestamp-based value that changes between test runs.
*/
export function normalizeItemReferences(dump: any): void {
const input = dump?.body?.input;
if (!Array.isArray(input)) {
return;
}
let refIndex = 0;
for (const item of input) {
if (item?.type === "item_reference" && item?.id) {
item.id = `[[ITEM_REF_${refIndex}]]`;
refIndex++;
}
}
}
/**
* Normalizes tool_call IDs and tool_call_id references to be deterministic.
* Tool call IDs have the format "call_[timestamp]_[index]" which changes between runs.
*/
export function normalizeToolCallIds(dump: any): void {
const messages = dump?.body?.messages;
if (!Array.isArray(messages)) {
return;
}
const oldToNewId: Record<string, string> = {};
let toolCallIndex = 0;
// First pass: collect all tool_call IDs and create mapping
for (const message of messages) {
if (message?.tool_calls && Array.isArray(message.tool_calls)) {
for (const toolCall of message.tool_calls) {
if (toolCall?.id && !oldToNewId[toolCall.id]) {
oldToNewId[toolCall.id] = `[[TOOL_CALL_${toolCallIndex}]]`;
toolCallIndex++;
}
}
}
}
// Second pass: replace all IDs
for (const message of messages) {
if (message?.tool_calls && Array.isArray(message.tool_calls)) {
for (const toolCall of message.tool_calls) {
if (toolCall?.id && oldToNewId[toolCall.id]) {
toolCall.id = oldToNewId[toolCall.id];
}
}
}
if (message?.tool_call_id && oldToNewId[message.tool_call_id]) {
message.tool_call_id = oldToNewId[message.tool_call_id];
}
}
}
/**
* Normalizes fileId hashes in versioned_files to be deterministic.
* FileIds are SHA-256 hashes that may include non-deterministic components
* like app paths with timestamps. This replaces them with stable placeholders
* based on content sorting.
*/
export function normalizeVersionedFiles(dump: any): void {
const vf = dump?.body?.dyad_options?.versioned_files;
if (!vf?.fileIdToContent) {
return;
}
const fileIdToContent = vf.fileIdToContent as Record<string, string>;
// Create mapping from old fileId to new deterministic fileId
// Sort by content to ensure deterministic ordering
const entries = Object.entries(fileIdToContent).sort((a, b) =>
String(a[1]).localeCompare(String(b[1])),
);
const oldToNewId: Record<string, string> = {};
const newFileIdToContent: Record<string, string> = {};
entries.forEach(([oldId, content], index) => {
const newId = `[[FILE_ID_${index}]]`;
oldToNewId[oldId] = newId;
newFileIdToContent[newId] = content;
});
vf.fileIdToContent = newFileIdToContent;
// Update fileReferences
if (vf.fileReferences) {
vf.fileReferences = vf.fileReferences.map((ref: any) => ({
...ref,
fileId: oldToNewId[ref.fileId] ?? ref.fileId,
}));
}
// Update messageIndexToFilePathToFileId
if (vf.messageIndexToFilePathToFileId) {
for (const pathToId of Object.values(
vf.messageIndexToFilePathToFileId as Record<
string,
Record<string, string>
>,
)) {
for (const [filePath, id] of Object.entries(pathToId)) {
pathToId[filePath] = oldToNewId[id] ?? id;
}
}
}
}
/**
* Normalizes path separators to always use forward slashes.
* Used for cross-platform consistency in tests.
*/
export function normalizePath(path: string): string {
return path.replace(/\\/g, "/");
}
# TypeScript Strict Mode (tsgo) # TypeScript Strict Mode (tsgo)
The pre-commit hook runs `tsgo` (via `npm run ts`), which is stricter than `tsc --noEmit`. For example, passing a `number` to a function typed `(str: string | null | undefined)` may pass `tsc` but fail `tsgo` with `TS2345: Argument of type 'number' is not assignable to parameter of type 'string'`. Always wrap with `String()` when converting numbers to string parameters. The pre-commit hook runs `tsgo` (via `npm run ts`), which is stricter than `tsc --noEmit`. For example, passing a `number` to a function typed `(str: string | null | undefined)` may pass `tsc` but fail `tsgo` with `TS2345: Argument of type 'number' is not assignable to parameter of type 'string'`. Always wrap with `String()` when converting numbers to string parameters.
## ES2020 target limitations
The project's `tsconfig.app.json` targets ES2020 with `lib: ["ES2020"]`. Methods introduced in ES2021+ (like `String.prototype.replaceAll`) are not available on the `string` type. If code uses `replaceAll`, it needs an `as any` cast to avoid `TS2550: Property 'replaceAll' does not exist on type 'string'`. Do not remove these casts without updating the tsconfig target.
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论