johannes/assistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(vela): mock push-to-talk transcript updates

Browse Source
This commit is contained in:
Johannes Kresner
2026-04-08 20:13:36 +02:00
parent 103bb11954
commit 98bcc543f5
8 changed files with 179 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/architecture.md
View File

@@ -36,7 +36,7 @@ The repository now includes separate runnable workspaces for the UI and gateway
- PWA enabled
- WebSocket client
The current implementation is a minimal SvelteKit app with a single voice-session shell page. The shipped UI can open and close a browser WebSocket connection to the gateway `/ws` endpoint, show explicit connection status (`not connected`, `connecting`, `connected`, `disconnected`, `error`), expose mic control shell interactions that emit placeholder `input_audio.append` / `input_audio.commit` events, trigger one deterministic mocked turn while connected, and render the mocked user transcript plus mocked assistant response for the active session. This remains a shell only: there is no real microphone capture, real provider integration, or audio playback yet.
The current implementation is a minimal SvelteKit app with a single voice-session shell page. The shipped UI can open and close a browser WebSocket connection to the gateway `/ws` endpoint, show explicit connection status (`not connected`, `connecting`, `connected`, `disconnected`, `error`), expose mic control shell interactions that emit placeholder `input_audio.append` / `input_audio.commit` events, trigger one deterministic mocked turn while connected, render deterministic placeholder partial/final transcripts for the push-to-talk shell, and render the mocked user transcript plus mocked assistant response for the existing mocked-turn path. This remains a shell only: there is no real microphone capture, real provider integration, or audio playback yet.
#### Responsibilities
@@ -105,6 +105,7 @@ The current implementation is a minimal Fastify service with `/`, `/health`, and
- WebSocket upgrades on `/ws` create an ephemeral session immediately
- the gateway sends `session.ready` followed by `session.state` (`idle`) when the socket is established
- valid minimal client events, including placeholder `input_audio.append` / `input_audio.commit`, can move the session between `idle` and `listening`
- placeholder `input_audio.append` emits deterministic mocked `transcript.partial` events and `input_audio.commit` emits one deterministic mocked `transcript.final`
- `mocked.turn.trigger` drives a fixed transcript/response event sequence over the existing shared protocol
- only one mocked turn is allowed in flight per session at a time
- invalid JSON, invalid envelopes, and malformed frames are handled defensively so the process stays up
@@ -115,12 +116,13 @@ The current implementation is a minimal Fastify service with `/`, `/health`, and
- exposes connect, disconnect, mic-control shell interactions, and mocked-turn controls
- does not request microphone permission or capture real microphone audio
- only emits placeholder `input_audio.append` / `input_audio.commit` events; it does not send real audio data or play back audio
- renders the latest placeholder partial transcript during a push-to-talk shell turn and replaces it with the final deterministic transcript on commit
- reads mocked transcript and mocked response events from the shared protocol contract
## Voice Pipeline
```text
Mic control shell / mocked turn button → Placeholder `input_audio.append` / `input_audio.commit` or mocked session flow → Transcript events → Response text events → UI
Mic control shell / mocked turn button → Placeholder `input_audio.append` / `input_audio.commit` or mocked session flow → Deterministic transcript events → Mocked response text events when using mocked.turn.trigger → UI
```
This mocked vertical slice intentionally stands in for the future real pipeline: