• Adekunle James Adeniji's avatar
    feat: add voice input feature with transcription support (#2344) · 31c1a145
    Adekunle James Adeniji 提交于
    closes #1804
    <!-- devin-review-badge-begin -->
    
    ---
    
    <a href="https://app.devin.ai/review/dyad-sh/dyad/pull/2344">
      <picture>
    <source media="(prefers-color-scheme: dark)"
    srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img
    src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1"
    alt="Open with Devin">
      </picture>
    </a>
    <!-- devin-review-badge-end -->
    
    <!-- CURSOR_SUMMARY -->
    ---
    
    > [!NOTE]
    > Introduces voice input across chat inputs with transcription via Dyad
    Engine and Pro gating.
    > 
    > - Replaces send row with `LexicalVoiceInputRow` in `ChatInput` and
    `HomeChatInput`, adding mic control, waveform (`VoiceWaveform`), and
    send/cancel integration
    > - New `VoiceInputButton` handles Pro-only disabled state,
    recording/transcribing states, and tooltips
    > - New hooks `useAudioRecorder` and `useVoiceInput` to record via
    `MediaRecorder`, visualize with `AnalyserNode`, and call
    `ipc.misc.transcribeAudio`
    > - IPC: adds `misc.transcribeAudio` contract, registers
    `transcription_handlers` that validate input, support E2E mock, and call
    `transcribeWithDyadEngine`
    > - Dyad Engine util: adds `transcribeWithDyadEngine` with request-id
    attempt tracking and multipart upload to `/audio/transcriptions`
    > - E2E tests for voice flow and Pro gating; mocks `getUserMedia` and
    asserts transcription append
    > 
    > <sup>Written by [Cursor
    Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
    7dc1944bf0149a9f88b63a3fdfe0df83e7aa4f9f. This will update automatically
    on new commits. Configure
    [here](https://cursor.com/dashboard?tab=bugbot).</sup>
    <!-- /CURSOR_SUMMARY -->
    
    <!-- This is an auto-generated description by cubic. -->
    ---
    ## Summary by cubic
    Adds voice input with waveform visualization and transcription for chat,
    gated to Dyad Pro users. Improves reliability with fixes for recording
    setup leaks, analyser state, and proper audio MIME typing so IPC-backed
    Dyad Engine transcription consistently appends text; addresses #1804.
    
    - **New Features**
    - Integrated VoiceInputButton and VoiceWaveform via LexicalVoiceInputRow
    in ChatInput and HomeChatInput; appends transcribed text to the input.
    - Added useAudioRecorder/useVoiceInput hooks to record via
    MediaRecorder, visualize with AnalyserNode, and invoke IPC channel
    chat:transcribe.
    - Pro gating with tooltip and disabled state for non-Pro users;
    recording can always be stopped.
    - IPC handler validates payloads and calls Dyad Engine via multipart
    upload; includes E2E mock support.
    - E2E tests mock getUserMedia and verify transcription append and
    Pro-only disabled state.
    
    - **Migration**
    - Provide a Dyad Pro API key (settings or DYAD_PRO_API_KEY) and enable
    Dyad Pro.
      - Ensure microphone permissions are granted.
    - Optionally set DYAD_ENGINE_URL; defaults to https://engine.dyad.sh/v1.
    
    <sup>Written for commit fa71433ae270a7276e5466c6c8df359eab1eb03d.
    Summary will update on new commits.</sup>
    
    <!-- End of auto-generated description by cubic. -->
    
    ---------
    Co-authored-by: 's avatarWill Chen <willchen90@gmail.com>
    31c1a145
名称
最后提交
最后更新
.agents 正在载入提交数据...
.claude 正在载入提交数据...
.cursor/rules 正在载入提交数据...
.devcontainer 正在载入提交数据...
.github 正在载入提交数据...
.husky 正在载入提交数据...
.storybook 正在载入提交数据...
assets 正在载入提交数据...
docs 正在载入提交数据...
drizzle 正在载入提交数据...
e2e-tests 正在载入提交数据...
makers 正在载入提交数据...
packages/@dyad-sh 正在载入提交数据...
plans 正在载入提交数据...
rules 正在载入提交数据...
scaffold 正在载入提交数据...
scripts 正在载入提交数据...
shared 正在载入提交数据...
src 正在载入提交数据...
testing 正在载入提交数据...
tools 正在载入提交数据...
worker 正在载入提交数据...
workers/tsc 正在载入提交数据...
.cursorignore 正在载入提交数据...
.env.example 正在载入提交数据...
.eslintrc.json 正在载入提交数据...
.gitattributes 正在载入提交数据...
.gitignore 正在载入提交数据...
.npmrc 正在载入提交数据...
.oxfmtrc.json 正在载入提交数据...
.oxlintrc.json 正在载入提交数据...
.prettierignore 正在载入提交数据...
.prettierrc 正在载入提交数据...
AGENTS.md 正在载入提交数据...
CLA.md 正在载入提交数据...
CLAUDE.md 正在载入提交数据...
CONTRIBUTING.md 正在载入提交数据...
LICENSE 正在载入提交数据...
README.md 正在载入提交数据...
SECURITY.md 正在载入提交数据...
biome.json 正在载入提交数据...
components.json 正在载入提交数据...
drizzle.config.ts 正在载入提交数据...
forge.config.ts 正在载入提交数据...
forge.env.d.ts 正在载入提交数据...
index.html 正在载入提交数据...
lint-staged.config.js 正在载入提交数据...
merge.config.ts 正在载入提交数据...
package-lock.json 正在载入提交数据...
package.json 正在载入提交数据...
playwright.config.ts 正在载入提交数据...
tsconfig.app.json 正在载入提交数据...
tsconfig.json 正在载入提交数据...
tsconfig.node.json 正在载入提交数据...
vite.main.config.mts 正在载入提交数据...
vite.preload.config.mts 正在载入提交数据...
vite.renderer.config.mts 正在载入提交数据...
vite.worker.config.mts 正在载入提交数据...
vitest.config.ts 正在载入提交数据...
windowsSign.ts 正在载入提交数据...