• Will Chen's avatar
    feat: add daily deflake-e2e-recent-prs skill and workflow (#2590) · 1e90ac6b
    Will Chen 提交于
    ## Summary
    - Add new `/dyad:deflake-e2e-recent-prs` command that automatically
    gathers flaky E2E tests from recent PR Playwright summary comments,
    ranks them by frequency, and deflakes them sequentially
    - Add `claude-deflake-e2e.yml` GitHub Actions workflow that runs daily
    at 2 AM PST on self-hosted macOS runners (with workflow_dispatch support
    for manual triggers)
    - Document the new command in `.claude/README.md`
    
    ## Test plan
    - Trigger the workflow manually via `gh workflow run
    claude-deflake-e2e.yml` and verify it correctly scans recent PRs for
    flaky tests and attempts to deflake them
    - Verify the cron schedule triggers at the expected time
    
    🤖 Generated with [Claude Code](https://claude.com/claude-code)
    <!-- devin-review-badge-begin -->
    
    ---
    
    <a href="https://app.devin.ai/review/dyad-sh/dyad/pull/2590"
    target="_blank">
      <picture>
    <source media="(prefers-color-scheme: dark)"
    srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img
    src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
    alt="Open with Devin">
      </picture>
    </a>
    <!-- devin-review-badge-end -->
    
    <!-- CURSOR_SUMMARY -->
    ---
    
    > [!NOTE]
    > **Medium Risk**
    > Adds a scheduled GitHub Action with write permissions that can open
    PRs, so misconfiguration or prompt issues could create noisy/unintended
    changes; however it’s limited to CI/automation and test-only guidance.
    > 
    > **Overview**
    > Adds a new Claude slash command, `/dyad:deflake-e2e-recent-prs`, that
    scans recent PR comments for Playwright “Flaky Tests”, ranks them by
    frequency, and runs deflaking steps per spec (including guidance to
    disable retries via `PLAYWRIGHT_RETRIES=0`) before optionally opening a
    fix PR.
    > 
    > Introduces a scheduled/manual GitHub Actions workflow
    (`claude-deflake-e2e.yml`) that runs daily on self-hosted macOS ARM64,
    installs dependencies/browsers, and invokes the new command via
    `anthropics/claude-code-action`. Documentation is updated to list the
    new command and the existing `/dyad:deflake-e2e` instructions are
    tightened to always disable Playwright retries.
    > 
    > <sup>Written by [Cursor
    Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
    10b9158363c6b9ae9a3f3ba52ad118149fb9cbd3. This will update automatically
    on new commits. Configure
    [here](https://cursor.com/dashboard?tab=bugbot).</sup>
    <!-- /CURSOR_SUMMARY -->
    
    <!-- This is an auto-generated description by cubic. -->
    ---
    ## Summary by cubic
    Adds a new /dyad:deflake-e2e-recent-prs command that scans recent PRs
    for Playwright flake reports, ranks flaky tests, and deflakes them. Adds
    a daily GitHub Action that runs it at 10:00 UTC (2 AM PST / 3 AM PDT) on
    self‑hosted macOS ARM64 to keep E2E tests stable.
    
    - **New Features**
    - Command scans recent PRs (default 20), parses Playwright summary
    comments from github-actions[bot], ranks by frequency, and deflakes
    specs sequentially; can push fixes via /dyad:pr-push.
    - New claude-deflake-e2e.yml workflow supports manual dispatch with
    pr_count, sets up Node/pnpm, installs Chromium, builds the fake LLM
    server, and runs the command via anthropics/claude-code-action.
    
    - **Bug Fixes**
    - Fixed spec path handling (no double .spec.ts), added gh api
    --paginate, switched to generic PR search, clarified {owner}/{repo} vs
    <pr_number>, and noted DST in the cron comment.
    - Disabled Playwright automatic retries in all deflake steps to prevent
    false passes, including debug and snapshot update commands.
    
    <sup>Written for commit 10b9158363c6b9ae9a3f3ba52ad118149fb9cbd3.
    Summary will update on new commits.</sup>
    
    <!-- End of auto-generated description by cubic. -->
    
    ---------
    Co-authored-by: 's avatarClaude Opus 4.5 <noreply@anthropic.com>
    1e90ac6b
deflake-e2e-recent-prs.md 5.0 KB