• Will Chen's avatar
    Improve test failure reporting with snapshot detection (#2484) · ea3b676f
    Will Chen 提交于
    ## Summary
    Enhanced the Playwright test summary generation to better handle
    snapshot failures and provide more actionable test commands in PR
    comments.
    
    ## Key Changes
    - **Added snapshot failure detection**: New `isSnapshotFailure()`
    function that identifies snapshot-related test failures by checking
    error messages for common snapshot comparison keywords
    - **Simplified test command output**: Removed the separate
    `generateCommands()` function and consolidated to a single command per
    test
    - **Smart command generation**: Automatically appends
    `--update-snapshots` flag only for snapshot failures, reducing confusion
    about when to use it
    - **Improved readability**: 
    - Reformatted test commands into a single code block with better
    organization
    - Added error preview (first 120 chars) to help developers understand
    failure context
    - Added `export PLAYWRIGHT_HTML_OPEN=never` to prevent browser windows
    from opening during test runs
    - Changed section description from "run or update snapshots" to "re-run
    failing tests locally" for clarity
    
    ## Implementation Details
    - The snapshot detection checks for 10 different snapshot-related error
    message patterns (case-insensitive)
    - Test commands are now grouped in a single bash code block for easier
    copy-pasting
    - Error messages are truncated to 120 characters to keep the comment
    concise while still providing useful context
    
    https://claude.ai/code/session_014DzYzvnt4559rZEGLwbVm5
    <!-- devin-review-badge-begin -->
    
    ---
    
    <a href="https://app.devin.ai/review/dyad-sh/dyad/pull/2484">
      <picture>
    <source media="(prefers-color-scheme: dark)"
    srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img
    src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
    alt="Open with Devin">
      </picture>
    </a>
    <!-- devin-review-badge-end -->
    
    <!-- This is an auto-generated description by cubic. -->
    ---
    ## Summary by cubic
    Improved Playwright PR test failure reporting by detecting snapshot
    mismatches and generating clearer, single-run commands. Makes it faster
    and safer to re-run failures locally and reduces confusion around
    snapshot updates.
    
    - **New Features**
    - Detect snapshot failures from error text and auto-append
    `--update-snapshots` only when needed.
    - Consolidate all failing test commands into one bash block with
    `PLAYWRIGHT_HTML_OPEN=never` set once.
    - Add a short error preview (first 120 chars) as a comment above each
    command.
    - Sanitize spec paths and escape test names/error previews to prevent
    injection.
    
    <sup>Written for commit 0515c892605dc303c60d1dbae4a2c0cd3a537481.
    Summary will update on new commits.</sup>
    
    <!-- End of auto-generated description by cubic. -->
    
    <!-- CURSOR_SUMMARY -->
    ---
    
    > [!NOTE]
    > **Low Risk**
    > Low risk: changes only affect CI-generated PR comments and local rerun
    command formatting, with added input sanitization reducing injection
    risk.
    > 
    > **Overview**
    > Improves the Playwright PR comment generator to **detect
    snapshot-related failures** and tailor rerun commands accordingly.
    > 
    > The macOS “Test Commands” section is reformatted into a single
    copy-pasteable bash block (including `PLAYWRIGHT_HTML_OPEN=never`), adds
    a short error preview per test, and appends `--update-snapshots` *only*
    when a failure looks snapshot-related. Command generation now also
    sanitizes the spec path and removes the separate run/update command
    helper.
    > 
    > <sup>Written by [Cursor
    Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
    0515c892605dc303c60d1dbae4a2c0cd3a537481. This will update automatically
    on new commits. Configure
    [here](https://cursor.com/dashboard?tab=bugbot).</sup>
    <!-- /CURSOR_SUMMARY -->
    
    ---------
    Co-authored-by: 's avatarClaude <noreply@anthropic.com>
    ea3b676f
名称
最后提交
最后更新
.claude 正在载入提交数据...
.cursor/rules 正在载入提交数据...
.devcontainer 正在载入提交数据...
.github 正在载入提交数据...
.husky 正在载入提交数据...
assets 正在载入提交数据...
docs 正在载入提交数据...
drizzle 正在载入提交数据...
e2e-tests 正在载入提交数据...
makers 正在载入提交数据...
packages/@dyad-sh 正在载入提交数据...
scaffold 正在载入提交数据...
scripts 正在载入提交数据...
shared 正在载入提交数据...
src 正在载入提交数据...
testing 正在载入提交数据...
tools 正在载入提交数据...
worker 正在载入提交数据...
workers/tsc 正在载入提交数据...
.cursorignore 正在载入提交数据...
.env.example 正在载入提交数据...
.eslintrc.json 正在载入提交数据...
.gitattributes 正在载入提交数据...
.gitignore 正在载入提交数据...
.oxfmtrc.json 正在载入提交数据...
.oxlintrc.json 正在载入提交数据...
.prettierignore 正在载入提交数据...
.prettierrc 正在载入提交数据...
AGENTS.md 正在载入提交数据...
CLA.md 正在载入提交数据...
CLAUDE.md 正在载入提交数据...
CONTRIBUTING.md 正在载入提交数据...
LICENSE 正在载入提交数据...
README.md 正在载入提交数据...
SECURITY.md 正在载入提交数据...
biome.json 正在载入提交数据...
components.json 正在载入提交数据...
drizzle.config.ts 正在载入提交数据...
forge.config.ts 正在载入提交数据...
forge.env.d.ts 正在载入提交数据...
index.html 正在载入提交数据...
lint-staged.config.js 正在载入提交数据...
merge.config.ts 正在载入提交数据...
package-lock.json 正在载入提交数据...
package.json 正在载入提交数据...
playwright.config.ts 正在载入提交数据...
tsconfig.app.json 正在载入提交数据...
tsconfig.json 正在载入提交数据...
tsconfig.node.json 正在载入提交数据...
vite.main.config.mts 正在载入提交数据...
vite.preload.config.mts 正在载入提交数据...
vite.renderer.config.mts 正在载入提交数据...
vite.worker.config.mts 正在载入提交数据...
vitest.config.ts 正在载入提交数据...
windowsSign.ts 正在载入提交数据...