Universal Windows App Development with Cortana and the Speech SDK 1 and 2 @Channel9

Course Website: http://channel9.msdn.com/Series/Universal-Windows-App-Development-with-Cortana-and-the-Speech-SDK

1. Introducing Cortana & Getting Started with Speech

Speech is Personal, Experiential.

Bing powers Cortana. Cloud recognizer runs algorithms

Core Functions: Communicate. Remember. Find.

Fun Cortana. Can tell jokes. Can sing a song.

2. Using Speech Synthesis in Mobile Apps

2.1 Basic Speech Synthesis Integration

2.1.1 Speech Synthesis Options

Speak with default speech settings. -Windows.Media.SpeechSynthesisnamespace

Speak with any installed speech language

Customize Text-to-Speech(TTS) voice with speech synthesis Markup Language (SSML)

TTS support introduced in WIndows Phone 8 SDK and Windows Store 8.1 SDK

2.1.2 In Application Speech Synthesis (Using default voice)

//windows phone store app

Speech Thesis uses a MediaElement control to speak text

// Synthesis<MediaElement Name="audioPlayer" AutoPlay="True".../>

C# code behind: //Speak text

//Fundtion to speak a text stringprivate async void SpeakText(MediaElement audioPlayer, string textToSpeak){SpeechSynthesizer synthesizer=new SpeechSynthesizer();SpeechSynthesisStream ttsStream= await synthesizer.SynthesizeTextToStreamAsync(textToSpeak);audioPlayer.SetSource(ttsStream,"");// This starts the player because AutoPlay="True"}

<strong>Demo to use default voice: </strong>

  private async void ButtonLookup_Click(object sender, RoutedEventArgs e)        {            string location = txtLocation.Text.Trim();            var wr = await owms.GetWeather(location);            if (wr != null)            {                var weatherText = "The current temperature in {0} is {1}°F, with a high today of {2}° and a low of {3}°.";                string weatherMessage = string.Format(weatherText, wr.Name, (int)wr.MainWeather.Temp, (int)wr.MainWeather.MaximumTemp, (int)wr.MainWeather.MinimumTemp);                lblMessage.Text = weatherMessage;                lblTemp.Text = string.Format("{0}°", (int)wr.MainWeather.Temp);                ReadText(weatherMessage);            }        }

 private async void ReadText(string mytext)        {            //Reminder: You need to enable the Microphone capabilitiy in Windows Phone projects            //Reminder: Add this namespace in your using statements            //using Windows.Media.SpeechSynthesis;            // The media object for controlling and playing audio.            MediaElement mediaplayer = new MediaElement();            // The object for controlling the speech synthesis engine (voice).            using (var speech = new SpeechSynthesizer())            {                //Retrieve the first female voice                speech.Voice = SpeechSynthesizer.AllVoices                    .First(i => (i.Gender == VoiceGender.Female && i.Description.Contains("United States")));                // Generate the audio stream from plain text.                SpeechSynthesisStream stream = await speech.SynthesizeTextToStreamAsync(mytext);                // Send the stream to the media object.                mediaplayer.SetSource(stream, stream.ContentType);                mediaplayer.Play();            }        }

2.1.3 Required Capability

Microphone Capability in package.appxmanifest

-Required for TTS in Windows Phone apps

- Optional for TTS in Windows Store apps

<Capabilities>    <Capability Name="internetClientServer"/>    <DeviceCapability Name="microphone"/></Capabilities>

2.2 Speech Synthesis Voice Settings

VoiceInformation currentVoice=(VoiceInformation) lstVoice.SelectedItem;speech.voice=currentVoice;SpeechSynthesisStream stream=await speech.SynthesizeTextToStreamAsync(myText);mediaplayer.SetSource(stream,stream.ContentType);mediaplayer.Play();

2.3 Speech Synthesis Markup Language

<speak   version='1.0'  xmlns='http://www.w3.org/2001/10/synthesis'  xml:lang='en-us'>  <voice name='Microsoft Zira Mobile'>    <prosody pitch='low' rate='0.9'>      This is the text what will be read by the speech synthesizer.    <prosody>   <voice><speak>

private async void ReadSsmlText(string mytext){VoiceInformation currentVoice=(VoiceInformation)lstVoices.SelectedItem;string Ssml=@"<speak version='1.0' " +"xmlns='http://www.w3.org/2001/10/synthesis' xml:lang='" + currentVoice.Language+"'>"+"<voice name='"+ currentVoice.DisplayName+"'>"+"<prosody pitch='"+lstPitch.SelectedItem.ToString() +"' rate=' "+sldRtae.Value.ToString()+"' </voice></speak>";SpeechSynthesisStream stream=await speech.SynthesizeSsmlToStreamAsync(Ssml);mediaplayer.SetSource(stream,stream.ContentType);mediaplayer.Play();}

2.4 Advanced Text-to-Speech Topics

2.4.1 Save Speech Audio Streams to a File

//open the output streamWidnows.Storage.Streams.Buffer buffer =new Windows.Storage.Streams.Buffer(4096);IRandomAccessStream writeStream=(IRandomAccessStream) await file.OpenAsync(FileAccessMode.ReadWrite);IOutputStream outputStream = writeStream.GetOutputStreamAt(0);DataWriter dataWriter =new DataWriter(outputStream);//copy the stream data into the filewhile(synthesisStream.Position<synthesisStream.Size){await synthesisStream.ReadAsync(buffer,4096,InputStreamOptions.None);dataWriter.WriterBuffer(buffer);}

2.4.2 Play Speech Audio in the Background

MediaElement can play in the background

 

AudioCategory="BackgroundCapableMedia"    for Windows Store 8.1 only

Windows phone 8.1 requires a background audio agent

SystemMediaControls