Johannes Kresner
bba0095bc0
Establish the monorepo, tooling, and starter apps so UI and gateway development can begin from a documented, runnable baseline.
98 lines
1.6 KiB
Markdown
98 lines
1.6 KiB
Markdown
# Vela Integrations and Tool Safety
|
|
|
|
## Current Runtime Baseline
|
|
|
|
- `vela-ui` is implemented as a SvelteKit application
|
|
- `vela-gateway` is implemented as a Fastify service
|
|
- current integration work beyond the gateway HTTP baseline remains future implementation
|
|
|
|
## STT (Speech-to-Text)
|
|
|
|
### Primary Option
|
|
|
|
- `whisper.cpp`
|
|
|
|
### Deployment
|
|
|
|
- start on NanoPi
|
|
- move to NAS if latency is insufficient
|
|
|
|
### Requirements
|
|
|
|
- streaming transcription
|
|
- partial and final output
|
|
- low latency, with sub-second response preferred
|
|
|
|
## TTS (Text-to-Speech)
|
|
|
|
### Engine
|
|
|
|
- Kokoro TTS
|
|
|
|
### Deployment
|
|
|
|
- prefer NAS for more compute headroom
|
|
|
|
### API Contract
|
|
|
|
```http
|
|
POST /speak
|
|
{
|
|
"text": "...",
|
|
"voice": "vela",
|
|
"format": "wav"
|
|
}
|
|
```
|
|
|
|
### Requirements
|
|
|
|
- streaming audio preferred
|
|
- low startup latency
|
|
- interrupt support
|
|
|
|
## Tool System
|
|
|
|
### Home Assistant Tool
|
|
|
|
#### Functions
|
|
|
|
```ts
|
|
turn_on(entity_id);
|
|
turn_off(entity_id);
|
|
set_temperature(entity_id, value);
|
|
get_state(entity_id);
|
|
```
|
|
|
|
#### Backend
|
|
|
|
- REST API
|
|
- optional Conversation API
|
|
|
|
#### Safety
|
|
|
|
- require confirmation for destructive actions
|
|
- require confirmation for irreversible or significant state changes
|
|
- keep secrets server-side only
|
|
|
|
### SearXNG Tool
|
|
|
|
#### Endpoint
|
|
|
|
```http
|
|
GET /search?q=...&format=json
|
|
```
|
|
|
|
#### Flow
|
|
|
|
- query SearXNG
|
|
- return top results
|
|
- let the LLM summarize the result set
|
|
|
|
## Safety Rules
|
|
|
|
- the LLM does not directly control systems
|
|
- all external actions go through explicit tool adapters
|
|
- Home Assistant write actions require confirmation
|
|
- frontend must not contain Home Assistant tokens or other secrets
|
|
- ambiguous tool intents should be clarified instead of guessed
|