Guides

Voice Design

Generate entirely new voices simply by describing their characteristics, age, and accent.

Voice Design mode lets you synthesize speech in entirely new voices simply by describing their characteristics. Unlike traditional cloning, you do not need to upload or record reference audio. Instead, you supply speaker attributes (the instruct parameter) such as gender, age, pitch, and accent, and the backend generates a matching voice on the fly.

Quick Example#

You can generate designed voices programmatically. Passing a comma-separated list of attributes to the instruct parameter builds the voice identity.

Python

import torch
from omnivoice import OmniVoice

model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)

audio = model.generate(
    text="This is a test of dynamically designed speech synthesis.",
    instruct="female, young adult, high pitch, british accent",
)

Supported Attributes#

The standard Voice Design taxonomy covers six primary speaker categories. You can specify one attribute from each category; the model automatically fills in any omitted attributes.

Gender#

English: male, female
Chinese: 男, 女

Age#

English: child, teenager, young adult, middle-aged, elderly
Chinese: 儿童, 少年, 青年, 中年, 老年

Pitch#

English: very low pitch, low pitch, moderate pitch, high pitch, very high pitch
Chinese: 极低音调, 低音调, 中音调, 高音调, 极高音调

Style#

English: whisper
Chinese: 耳语

whisper is the primary delivery style accepted by the base taxonomy. Expression tags and emotions are managed differently depending on the active speech engine.

English Accent#

English accents are only active when the synthesis text is in English:

american accent
british accent
australian accent
canadian accent
indian accent
chinese accent
korean accent
japanese accent
portuguese accent
russian accent

Chinese Dialect#

Chinese dialects are only active when the synthesis text is in Chinese:

河南话 (Henan)
陕西话 (Shaanxi)
四川话 (Sichuan)
贵州话 (Guizhou)
云南话 (Yunnan)
桂林话 (Guilin)
济南话 (Jinan)
石家庄话 (Shijiazhuang)
甘肃话 (Gansu)
宁夏话 (Ningxia)
青岛话 (Qingdao)
东北话 (Dongbei)

Natural Language Description Mapping#

To simplify voice creation, the backend provides a deterministic keyword mapper that projects natural-language descriptions (e.g., "a warm elderly British storyteller, slightly raspy") onto the structured taxonomy. This parsing is executed locally on the CPU without requiring network calls or running neural models.

Description Parser Endpoint#

You can parse a description string by sending a request to the local API:

POST /design/describe

Request Payload:

JSON

{
  "description": "a female teenager with a very low pitch speaking with a sichuan dialect"
}

Response Payload:

JSON

{
  "attrs": {
    "Gender": "female",
    "Age": "teenager",
    "Pitch": "very low pitch",
    "Style": "Auto",
    "EnglishAccent": "Auto",
    "ChineseDialect": "四川话"
  },
  "instruct": "female, teenager, very low pitch, 四川话",
  "matched": [
    { "category": "Gender", "token": "female", "phrase": "female" },
    { "category": "Age", "token": "teenager", "phrase": "teenager" },
    { "category": "Pitch", "token": "very low pitch", "phrase": "very low pitch" },
    { "category": "ChineseDialect", "token": "四川话", "phrase": "sichuan" }
  ],
  "unmatched": [
    "with a",
    "speaking with a"
  ]
}

Mapping Logic Rules#

Numeric Age Parsing: Explicit ages in descriptions (such as "25yo", "35 years old", or "70-year-old") are automatically mapped to taxonomic brackets:
- $\le 12$ years: child
- $13$ to $19$ years: teenager
- $20$ to $39$ years: young adult
- $40$ to $64$ years: middle-aged
- $\ge 65$ years: elderly
Specific Overrides: Rules check specific phrases before generic ones. For example, "young child" maps to child (rather than combining "young" and "child" into young adult), and "very deep" maps to very low pitch (instead of stopping at "deep" for low pitch).
Pinyin Support: Romanized names for Chinese dialects (like "dongbei", "szechuan", or "shaanxi") are translated into their native dialect characters.
Conflict Resolution: English accents and Chinese dialects cannot be applied simultaneously. If both are detected, the dialect takes priority and the English accent is reset to Auto.
Transparency: Words that do not map to the taxonomy (such as "raspy" or "storyteller") are returned in the unmatched array so the application can communicate which parts of the text were skipped.

Engine Support and Compatibility#

How Voice Design behaves depends on the capabilities of your active model. For details on how to configure and switch model backends, see Models & Hardware.

Tag-Based Engines (`instruct_vocabulary: "tags"`)#

The OmniVoice family models use the strict, structured tag taxonomy described above. Input strings are automatically sanitized and normalized.

Freeform Engines (`instruct_vocabulary: "freeform"`)#

Engines such as Qwen3-TTS VoiceDesign (MLX-Audio), VoxCPM2, and audio.cpp process description instructions as free-form text. Descriptions are passed raw to the model, allowing traits outside the predefined taxonomy (such as "raspy" or "breathy") to influence the voice timbre.

Unsupported Engines (`supports_voice_design: false`)#

Models like IndexTTS2, MOSS-TTS, dots.tts, Confucius4, GPT-SoVITS, Supertonic-3, and OmniVoice GGUF do not support voice design attributes. They require a physical reference audio clip or a preset voice.

If an unsupported engine is selected, the application interface directs you to the voice settings page.
A direct API /generate request with an instruct parameter sent to these engines will fail with a 422 Unprocessable Entity status code.
However, if you design a voice using a supported engine, generate a preview, and save it to your profile, unsupported engines can still replicate the voice by cloning that saved preview sample. See Voice Cloning & Gallery.

Saving and Exporting Personas (.ovsvoice)#

Designed voices are defined by their target attributes and a generation seed. Saving a design stores these parameters to your local profile database.

A seed remains fixed while you tweak attributes to keep the voice identity stable. Generating a "New Seed" alters the baseline speaker identity. Saved profiles can be exported and imported as portable voice files using the .ovsvoice persona bundle format.

Persona API Endpoints#

POST /personas/export/{profile_id}: Packages a voice profile and its associated audio references (including optional tags, licensing information, and preview tracks) into a downloadable .ovsvoice ZIP bundle.
POST /personas/import: Uploads and extracts a .ovsvoice (or legacy .omnivoice) package. The system registers the profile and extracts reference audio files to your local voice directory.
POST /personas/inspect: Reads a bundle's manifest, tags, licensing, and consent summary without writing files or making database updates.

During bundle import, the system enforces secure verification criteria:

The system checks for a signed attestation file (consent.json) and a recording (consent_audio) meeting minimum byte thresholds.
If valid consent text and audio are verified, the profile is marked with verified_own_voice: true.
If the audio payload is missing, too short, or lacks explicit consent text, the profile is registered but remains unverified.