feat(vela): add mocked turn transcript response slice
This commit is contained in:
@@ -36,13 +36,15 @@ The repository now includes separate runnable workspaces for the UI and gateway
|
||||
- PWA enabled
|
||||
- WebSocket client
|
||||
|
||||
The current implementation is a minimal SvelteKit app with a single voice-session shell page. The shipped UI can open and close a browser WebSocket connection to the gateway `/ws` endpoint, show explicit connection status (`not connected`, `connecting`, `connected`, `disconnected`, `error`), and surface session metadata for developers. Microphone capture, transcript rendering, interrupt controls, streamed assistant response display, and audio playback are not part of the current shell and remain future work.
|
||||
The current implementation is a minimal SvelteKit app with a single voice-session shell page. The shipped UI can open and close a browser WebSocket connection to the gateway `/ws` endpoint, show explicit connection status (`not connected`, `connecting`, `connected`, `disconnected`, `error`), trigger one deterministic mocked turn while connected, and render the mocked user transcript plus mocked assistant response for the active session. Microphone capture, real provider integration, and audio playback are still future work.
|
||||
|
||||
#### Responsibilities
|
||||
|
||||
Current shell responsibilities:
|
||||
|
||||
- connection state rendering
|
||||
- mocked-turn trigger rendering with disconnected/in-flight guards
|
||||
- mocked transcript and mocked assistant response rendering
|
||||
- developer-oriented session metadata rendering
|
||||
- browser session connect/disconnect controls
|
||||
|
||||
@@ -60,7 +62,10 @@ Current shell:
|
||||
- developer-focused voice-session panel
|
||||
- connect button
|
||||
- disconnect button
|
||||
- mocked-turn button
|
||||
- connection status indicator
|
||||
- mocked transcript display
|
||||
- mocked assistant response display
|
||||
- session metadata display
|
||||
|
||||
Future interactive voice screen:
|
||||
@@ -82,7 +87,7 @@ Future interactive voice screen:
|
||||
- Fastify (Node)
|
||||
- WebSocket-based session layer
|
||||
|
||||
The current implementation is a minimal Fastify service with `/`, `/health`, and a documented `/ws` WebSocket session endpoint. The gateway keeps one ephemeral in-memory session record per live socket connection and removes it on disconnect.
|
||||
The current implementation is a minimal Fastify service with `/`, `/health`, and a documented `/ws` WebSocket session endpoint. The gateway keeps one ephemeral in-memory session record per live socket connection, removes it on disconnect, and can run one deterministic mocked turn per session without involving any external providers.
|
||||
|
||||
#### Responsibilities
|
||||
|
||||
@@ -100,18 +105,26 @@ The current implementation is a minimal Fastify service with `/`, `/health`, and
|
||||
- WebSocket upgrades on `/ws` create an ephemeral session immediately
|
||||
- the gateway sends `session.ready` followed by `session.state` (`idle`) when the socket is established
|
||||
- valid minimal client events can move the session between `idle` and `listening`
|
||||
- `mocked.turn.trigger` drives a fixed transcript/response event sequence over the existing shared protocol
|
||||
- only one mocked turn is allowed in flight per session at a time
|
||||
- invalid JSON, invalid envelopes, and malformed frames are handled defensively so the process stays up
|
||||
|
||||
### Current UI shell behavior
|
||||
|
||||
- renders a minimal developer-focused voice-session panel
|
||||
- exposes connect and disconnect controls only
|
||||
- exposes connect, disconnect, and mocked-turn controls
|
||||
- does not request microphone permission
|
||||
- does not send or process audio data
|
||||
- reads `session.ready`, `session.state`, and `error` messages from the shared protocol contract
|
||||
- reads mocked transcript and mocked response events from the shared protocol contract
|
||||
|
||||
## Voice Pipeline
|
||||
|
||||
```text
|
||||
Mocked turn button → Gateway mocked session flow → Transcript events → Response text events → UI
|
||||
```
|
||||
|
||||
This mocked vertical slice intentionally stands in for the future real pipeline:
|
||||
|
||||
```text
|
||||
Mic → Gateway → STT → Transcript
|
||||
→ LLM → Tool Calls → Results
|
||||
|
||||
@@ -34,10 +34,12 @@ Prove the end-to-end interaction model with mocked or stubbed providers.
|
||||
- [x] bootstrap `vela-ui` as a runnable SvelteKit app in the Yarn workspace
|
||||
- [x] bootstrap `vela-gateway` as a runnable Fastify app in the Yarn workspace
|
||||
- [x] add the first UI voice-session shell with connect/disconnect controls and explicit WebSocket status
|
||||
- create a minimal UI with mic control, transcript, and response text
|
||||
- [x] create a minimal mocked-turn UI with transcript and response text over the shared WebSocket session
|
||||
- create a minimal UI with mic control
|
||||
- [x] create a gateway WebSocket session skeleton
|
||||
- implement mocked STT flow for partial and final transcript events
|
||||
- implement mocked LLM response streaming
|
||||
- [x] implement a mocked transcript/response vertical slice over the existing WebSocket session
|
||||
- implement mocked STT flow for partial transcript events
|
||||
- implement mocked LLM response streaming beyond the fixed deterministic slice
|
||||
- implement stubbed audio playback or placeholder TTS output
|
||||
- implement interrupt handling across the mocked pipeline
|
||||
|
||||
@@ -181,7 +183,10 @@ Polish the system after the core voice loop is reliable.
|
||||
|
||||
- `apps/vela-ui` now boots as a minimal SvelteKit app with a starter page
|
||||
- `apps/vela-ui` now includes a minimal voice-session shell that can connect to the gateway `/ws` endpoint and display developer-visible session status
|
||||
- `apps/vela-ui` can now trigger one deterministic mocked turn while connected and render the mocked transcript plus assistant response for the active session
|
||||
- `apps/vela-ui` now includes browser-level coverage for the mocked transcript/response slice, including connect, disconnect, and disconnected-state trigger guarding
|
||||
- `apps/vela-gateway` now boots as a minimal Fastify app with `/` and `/health` endpoints
|
||||
- `apps/vela-gateway` now exposes a minimal `/ws` WebSocket session skeleton with ephemeral in-memory sessions and defensive message handling
|
||||
- `apps/vela-gateway` now accepts `mocked.turn.trigger` and emits protocol-valid mocked transcript/response events with one in-flight mocked turn per session
|
||||
- `apps/vela-protocol` now provides the shared WebSocket event contract for the UI and gateway
|
||||
- backend framework choice is now concrete: Fastify
|
||||
|
||||
@@ -15,7 +15,7 @@ Current UI baseline:
|
||||
|
||||
- the browser opens a WebSocket directly to `/ws`
|
||||
- the UI tracks connection status separately from gateway session status
|
||||
- the UI currently consumes server events but does not send `session.start` or any audio events yet
|
||||
- the UI can send `mocked.turn.trigger` after `session.ready` while connected to request one deterministic mocked turn for the active session
|
||||
|
||||
## WebSocket Message Envelope
|
||||
|
||||
@@ -40,6 +40,7 @@ This increment intentionally keeps the envelope minimal:
|
||||
```ts
|
||||
type ClientEvent =
|
||||
| { type: "session.start"; payload: {} }
|
||||
| { type: "mocked.turn.trigger"; payload: {} }
|
||||
| { type: "input_audio.append"; payload: { chunk: string } }
|
||||
| { type: "input_audio.commit"; payload: {} }
|
||||
| { type: "response.cancel"; payload: {} };
|
||||
@@ -48,6 +49,7 @@ type ClientEvent =
|
||||
#### Client event intent
|
||||
|
||||
- `session.start` initializes a voice session without locking in transport or auth details yet
|
||||
- `mocked.turn.trigger` asks the gateway to run one obviously mocked, deterministic transcript/response turn
|
||||
- `input_audio.append` carries a chunk of captured input audio as an encoded string
|
||||
- `input_audio.commit` marks the current buffered user turn as ready for downstream processing
|
||||
- `response.cancel` interrupts the active listen/think/speak flow
|
||||
@@ -56,9 +58,12 @@ type ClientEvent =
|
||||
|
||||
- on connect, the gateway creates an ephemeral in-memory session and emits `session.ready` plus `session.state`
|
||||
- `session.start` is accepted as an idempotent session acknowledgment and re-sends readiness/state
|
||||
- `mocked.turn.trigger` is accepted only when no other mocked turn is already in flight for that session
|
||||
- a mocked turn emits deterministic `transcript.final`, `response.text.delta`, `response.completed`, and `session.state` events in protocol-valid order
|
||||
- `input_audio.append` updates the ephemeral session record and moves the session to `listening`
|
||||
- `input_audio.commit` resets the minimal buffered state and returns the session to `idle`
|
||||
- `response.cancel` resets the minimal session state back to `idle`
|
||||
- a second mocked-turn trigger during an active mocked turn produces `error` with code `mocked_turn_in_flight`
|
||||
- malformed JSON produces `error` with code `invalid_json`
|
||||
- invalid envelopes or unsupported client event names produce `error` with code `invalid_message`
|
||||
- malformed WebSocket frames are rejected without crashing the gateway process
|
||||
@@ -79,6 +84,7 @@ Notes:
|
||||
|
||||
- this UI state is transport-oriented and is separate from the shared gateway `session.state` payload
|
||||
- `session.state` currently reflects the gateway session phase (`idle`, `listening`, `thinking`, `speaking`)
|
||||
- the UI disables the mocked-turn control until `session.ready` arrives, while disconnected, or while a mocked turn is already in flight
|
||||
- the UI treats malformed server messages, browser WebSocket errors, and gateway `error` events as safe error states instead of throwing
|
||||
|
||||
### Server → Client
|
||||
@@ -109,6 +115,27 @@ type ServerEvent =
|
||||
- `response.completed` marks the current assistant turn as done
|
||||
- `error` is the minimal recoverable failure shape for both UI and gateway work
|
||||
|
||||
### Deterministic mocked turn sequence
|
||||
|
||||
For this increment, `mocked.turn.trigger` produces one fixed interaction for the active session:
|
||||
|
||||
```text
|
||||
session.state(listening)
|
||||
→ transcript.final("[mocked user] What is the current mocked vertical slice?")
|
||||
→ session.state(thinking)
|
||||
→ session.state(speaking)
|
||||
→ response.text.delta("[mocked assistant] ")
|
||||
→ response.text.delta("This is a deterministic mocked response from the gateway vertical slice.")
|
||||
→ response.completed
|
||||
→ session.state(idle)
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- the content is intentionally fixed and obviously mocked
|
||||
- no audio, STT, LLM, TTS, or external providers participate in this flow
|
||||
- `response.cancel` can stop the mocked turn early and return the session to `idle`
|
||||
|
||||
## Contract Scope for This Increment
|
||||
|
||||
This contract is intentionally limited to the smallest event set needed to unblock:
|
||||
@@ -118,6 +145,7 @@ This contract is intentionally limited to the smallest event set needed to unblo
|
||||
|
||||
Explicitly deferred for later increments:
|
||||
|
||||
- freeform typed user input
|
||||
- tool-calling events
|
||||
- streamed TTS/output-audio events
|
||||
- reconnect/resume semantics
|
||||
|
||||
@@ -61,13 +61,14 @@ Additional verification commands:
|
||||
mise exec -- yarn check:ui
|
||||
mise exec -- yarn build:ui
|
||||
mise exec -- yarn build:gateway
|
||||
npm run test --workspace vela-ui
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- the concrete framework choices are now SvelteKit for `vela-ui` and Fastify for `vela-gateway`
|
||||
- the UI is intentionally minimal and currently includes only a developer-facing WebSocket voice-session shell
|
||||
- the UI does not yet include mic capture, transcript rendering, assistant response rendering, or audio playback
|
||||
- the UI now renders a deterministic mocked transcript and assistant response flow, but does not yet include mic capture or audio playback
|
||||
- the gateway now exposes the minimal shared-protocol `/ws` WebSocket contract used by that shell
|
||||
- if your shell is configured for mise activation, plain `yarn` commands can be used after `mise install`
|
||||
- update this document when the repo layout or package manager workflow changes
|
||||
|
||||
Reference in New Issue
Block a user