/xnaSynth

Speech Synthesis for XNA

http://www.brains-N-brawn.com/xnaSynth 2/19/2007 casey chesnut

comment(s) 

Introduction

been mainly concentrating on speech-related programming lately. then XNA was released and i just had to write a program to run on my XBox 360 ... so i decided to combine the two and write a speech synthesis engine for XNA. so this article is an overiew of the xnaSynth game library. the code could also be useful to learn the basics of writing a speech synthesis engine.

had written a basic speech synthesis engine a couple years ago with the /ttSpeech article. the code is mostly written from scratch, but it uses resources from the open source Java project FreeTTS. specifically, it uses the CMU Dictionary for looking up phonemes for common english words, as well as the WAV diphones used for the voice. for this update, i started with that base and added some more functionality. also changed some pieces to make it work with XNA.

 

API

its really easy to use the xnaSynth in a game. first, you have to create a Speech Synthesizer during Initialize()

private SpeechSynthesizer speechSynthesizer = null;
//Initialize()
speechSynthesizer = new SpeechSynthesizer();

after that, you just have to call a couple methods to make it speak during Update().

//Update()
speechSynthesizer.SynthesizeText("hello world");
speechSynthesizer.Update();

to stop speech immediately, call :

speechSynthesizer.Clear();

there are also some overloads for more advanced usage :

public SynthesizedText SynthesizeText(string text, bool addToQueue, bool spellItOut)
public void SynthesizePhonemes(string text, string phonemes, bool addToQueue)

addToQueue - will queue the speech to be synthesized, otherwise it will interrupt the current speech

spellItOut - will spell out a word, instead of speaking it

SythesizePhonemes() - lets you pass in the specific phonemes to be spoken

Overview

the speech synthesis can be broken down into the following steps :

Limitations

ran into a couple limitations with XNA :

Sample

Conclusion

it wasn't too hard to create a Speech Synthesis engine for the XBox 360. XNA was a bit tricky to work with, but there weren't any problems that couldn't be worked around. can't wait to see how it improves in the future.

ultimately i hope that we get Microsoft's SAPI engine supported on the XBox 360, because their 'Microsoft Anna' voice sounds excellent. not to mention their extensive language support. and i'd really like to have more speech recognition support in games.

Source

here is the code for Windows and XBox 360. it also includes a slightly modified version of XGameLineInput

Updates

if i get time to write a game, then i'll probably update the API to provide more state info

Future

doing a user group presentation on Spech next. not sure what to play with after that ... later