Unverified 提交 0f2a68c7 authored 作者: Will Chen's avatar Will Chen 提交者: GitHub

fix: deflake E2E tests (setup_flow, chat_history, Navigation) (#2681)

- setup_flow.spec.ts: Wait for domcontentloaded after reload and increase timeout for "Install Node.js Runtime" button visibility check - chat_history.spec.ts: Add small delay between keyboard navigation events and increase timeout on class assertion to prevent ArrowUp race condition - Navigation.ts: Wait for "Apps" link to be visible before clicking in goToAppsTab() to prevent timeouts when sidebar loads slowly <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/dyad-sh/dyad/pull/2681" target="_blank"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1"> <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open with Devin"> </picture> </a> <!-- devin-review-badge-end --> --------- Co-authored-by: 's avatarclaude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: 's avatarClaude Opus 4.6 <noreply@anthropic.com> Co-authored-by: 's avatargemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
上级 f2541b6b
...@@ -41,8 +41,10 @@ test("should open, navigate, and select from history menu", async ({ po }) => { ...@@ -41,8 +41,10 @@ test("should open, navigate, and select from history menu", async ({ po }) => {
await expect(firstItem).toContainText("First test message"); await expect(firstItem).toContainText("First test message");
// Navigate up again to wrap to last item (newest message) // Navigate up again to wrap to last item (newest message)
// Wait briefly to ensure the previous navigation state has settled
await po.page.waitForTimeout(100);
await po.page.keyboard.press("ArrowUp"); await po.page.keyboard.press("ArrowUp");
await expect(lastItem).toHaveClass(/bg-accent/); await expect(lastItem).toHaveClass(/bg-accent/, { timeout: Timeout.MEDIUM });
// Select with Enter (selects newest message) // Select with Enter (selects newest message)
await po.page.keyboard.press("Enter"); await po.page.keyboard.press("Enter");
......
...@@ -17,7 +17,9 @@ export class Navigation { ...@@ -17,7 +17,9 @@ export class Navigation {
} }
async goToAppsTab() { async goToAppsTab() {
await this.page.getByRole("link", { name: "Apps" }).click(); const appsLink = this.page.getByRole("link", { name: "Apps" });
await expect(appsLink).toBeVisible({ timeout: 60000 });
await appsLink.click();
await expect(this.page.getByText("Build a new app")).toBeVisible(); await expect(this.page.getByText("Build a new app")).toBeVisible();
} }
......
...@@ -40,6 +40,7 @@ testSetup.describe("Setup Flow", () => { ...@@ -40,6 +40,7 @@ testSetup.describe("Setup Flow", () => {
// Start with Node.js not installed // Start with Node.js not installed
await po.setNodeMock(false); await po.setNodeMock(false);
await po.page.reload(); await po.page.reload();
await po.page.waitForLoadState("domcontentloaded");
// Verify setup banner and install button are visible // Verify setup banner and install button are visible
await expect( await expect(
...@@ -47,7 +48,7 @@ testSetup.describe("Setup Flow", () => { ...@@ -47,7 +48,7 @@ testSetup.describe("Setup Flow", () => {
).toBeVisible(); ).toBeVisible();
await expect( await expect(
po.page.getByRole("button", { name: "Install Node.js Runtime" }), po.page.getByRole("button", { name: "Install Node.js Runtime" }),
).toBeVisible(); ).toBeVisible({ timeout: 10000 });
// Manual configuration link should be visible // Manual configuration link should be visible
await expect( await expect(
......
...@@ -97,6 +97,13 @@ Each parallel Playwright worker gets its own fake LLM server on port `FAKE_LLM_B ...@@ -97,6 +97,13 @@ Each parallel Playwright worker gets its own fake LLM server on port `FAKE_LLM_B
When adding new test server URLs, update **both** the test fixtures (`e2e-tests/helpers/fixtures.ts`) and the Electron app source that consumes them. The app reads `process.env.FAKE_LLM_PORT` to build its `TEST_SERVER_BASE` URL — if you hardcode a port in app source, parallel workers will all hit the same server. When adding new test server URLs, update **both** the test fixtures (`e2e-tests/helpers/fixtures.ts`) and the Electron app source that consumes them. The app reads `process.env.FAKE_LLM_PORT` to build its `TEST_SERVER_BASE` URL — if you hardcode a port in app source, parallel workers will all hit the same server.
## Common flaky test patterns and fixes
- **After `page.reload()`**: Always add `await page.waitForLoadState("domcontentloaded")` before interacting with elements. Without this, the page may not have re-rendered yet.
- **Keyboard navigation events (ArrowUp/ArrowDown)**: Add `await page.waitForTimeout(100)` between sequential keyboard presses to let the UI state settle. Rapid keypresses can cause race conditions in menu navigation.
- **Navigation to tabs**: Use `await expect(link).toBeVisible({ timeout: Timeout.EXTRA_LONG })` before clicking tab links (especially in `goToAppsTab()`). Electron sidebar links can take time to render during app initialization.
- **Confirming flakiness**: Use `PLAYWRIGHT_RETRIES=0 PLAYWRIGHT_HTML_OPEN=never npm run e2e -- e2e-tests/<spec> --repeat-each=10` to reproduce flaky tests. `PLAYWRIGHT_RETRIES=0` is critical — CI defaults to 2 retries, hiding flakiness.
## E2E test fixtures with .dyad directories ## E2E test fixtures with .dyad directories
When adding E2E test fixtures that need a `.dyad` directory for testing: When adding E2E test fixtures that need a `.dyad` directory for testing:
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论