Universal Windows App Development with Cortana and the Speech SDK 3

3 Cortana Integration & Voice Commands

3.1 Create the Voice Command Definition (VCD) file

VCD file:

<?xml version="1.0" encoding="utf-8"?><!-- Be sure to use the new v1.1 namespace to utilize the new PhraseTopic feature --><<strong>VoiceCommands</strong> xmlns="http://schemas.microsoft.com/voicecommands/1.1">    <!-- The CommandSet Name is used to programmatically access the CommandSet -->    <<strong>CommandSet</strong> xml:lang="en-us" Name="englishCommands">        <!-- The CommandPrefix provides an alternative to your full app name for invocation -->        <<strong>CommandPrefix</strong>> MSDN </CommandPrefix>        <!-- The CommandSet Example appears in the global help alongside your app name -->        <<strong>Example</strong>> find 'Windows Phone Voice Commands' </Example>        <<strong>Command</strong> Name="MSDNSearch">            <!-- The Command example appears in the drill-down help page for your app -->            <Example> find 'how to install CommandSets' </Example>            <!-- ListenFor elements provide ways to say the command, including references to             {PhraseLists} and {PhraseTopics} as well as [optional] words -->            <<strong>ListenFor</strong>> Search </ListenFor>            <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>            <ListenFor> Find <strong>{dictatedSearchTerms}</strong> </ListenFor>            <ListenFor> Find </ListenFor>          <!--Feedback provides the displayed and spoken text when your command is triggered -->            <<strong>Feedback</strong>> Searching MSDN... </Feedback>            <!-- Navigate specifies the desired page or invocation destination for the Command-->            <<strong>Navigate</strong> Target="MainPage.xaml" />        </Command>        <<strong>Command</strong> Name="MSDNNaturalLanguage">            <Example> I want to go to the Windows Phone Dev center </Example>            <ListenFor> {naturalLanguage} </ListenFor>            <Feedback> Starting MSDN... </Feedback>            <Navigate Target="MainPage.xaml" />        </Command>        <<strong>PhraseTopic</strong> Label="dictatedSearchTerms" Scenario="Search">            <Subject> MSDN </Subject>        </PhraseTopic>        <<strong>PhraseTopic</strong> Label="naturalLanguage" Scenario="Natural Language">            <Subject> MSDN </Subject>        </PhraseTopic>    </CommandSet></VoiceCommands>

Command Prefix : Your app's name or a simplified form of the name. Users can use either to address your application. For Cortana to recognize. 

Example : top level of what a user can say

Command : can add one to many commands. Contains: what the user says to Cortana. What Cortana says in response. What action Cortana will perform.

Listen For: ways to say the command

Phrase Topic: Scenario hints to guide accuracy improvements.

Recognized in cloud, not on device.

Significantly increased speech recognition accuracy

FeedBack : Cortana 做出的回应。可自定义。

3.2 Register the VCD file on APP Startup

JavaScript

C#:

private async void RegisterVoiceCommands(){Uri uriVoiceCommands=new Uri("ms-appx:///vcd.xml", UriKind.Absolute):StorageFile file=await StorageFile.GetFileFromApplicationUriAsync(uriVoiceCommands);await VoiceCommandManager.InstallCommandSetsFromStorageFileAsync(file);}

3.3 Handle Voice Command Activation

//Windows Runtime App on Windows Phone 8.1, inside OnActivated override in App Class

protected override void OnActivated(IActivatedEventArgs args){    // Was the app activated by a voice command?    if (args.Kind == Windows.ApplicationModel.Activation.ActivationKind.VoiceCommand)    {        var commandArgs = args as Windows.ApplicationModel.Activation.VoiceCommandActivatedEventArgs;        Windows.Media.SpeechRecognition.SpeechRecognitionResult speechRecognitionResult = commandArgs.Result;        // If so, get the name of the voice command, the actual text spoken, and the value of Command/Navigate@Target.        string voiceCommandName = speechRecognitionResult.RulePath[0];        string textSpoken = speechRecognitionResult.Text;        string navigationTarget = speechRecognitionResult.SemanticInterpretation.Properties["NavigationTarget"][0];        switch (voiceCommandName)        {            case "showWidgets":                if (textSpoken.Contains("today's specials"))                {                    // Code to show today's specials.                    // To do this, refactor OnLaunched into a method that you can call from both OnLaunched and OnActivated.                    // The navigation logic will resemble this: rootFrame.Navigate(typeof(<today' specials page class>), e.Arguments);                }                else if (textSpoken.Contains("best sellers"))                {                    // Code to show the best sellers.                }                break;            // Cases for other voice commands.            default:                // There is no match for the voice command name.                break;        }    }}

if(args.kind == ActivationKind.VoiceCommand){VoiceCommandActivatedEventArgs vcArgs= (VoiceCommandActivatedEventArgs) args;string voiceCommandName=vcArgs.Result.RulePath.First();//what command lauched the appswitch(voiceCoomandName) //navigate to right page for the voice command{case "FindText" :         rootFrame.Navigate(typeof(MSDN.FindText), vcArgs.Result);         break;case "nlpCommand":          rootFrame.Navigate(typeof(MSDN.NlpCommand), vcArgs.Result);          break;}}

// windows Runtime App on Windows Phone 8.1 , inside OnNavigatedTo in FindText.xaml.csprotected override void OnNavigatedTo( NavigationEventArgs e){//get recignition result from parameter passed in frame.Navigate callSpeechRecognitionResult vcResult=e.Parameter as SpeechRecognitionResult;if (vcResult!=null){string recoText=vcResult.Text;//store the semantic dictionary for later useIReadOnlyDictionary<string,IReadOnlyList<string>> sematics=vdResult.SemanticInterpretaion.Properties;string voiceCommandName=vdResult.RulePath.First();if( voiceCommandName== ""FindText"){//what did the user say, for named phrase topic or list "slots"// users add some search termsif (semantics.ContainsKey("dictatedSearchTerms")){HandleFindTextWithSearchTerms (semantics["dictatedSearchTerms"][0]);}else{HandleNoSearchTerms();}}//else handle other voice commands}navigationHelper.OnNavigatedTo(e);}

//Windows Runtime App on Windows Phone 8.1, inside OnNavigatedTo in NlpCommand.xaml.cs

// free formatprotected override void onNavigatedTo (NavigationEventArgs e){base.OnNavigatedTo(e);//get recognition result from parameter passed in frame.Navigate callSpeechRecognitionResult vcResult=e.Parameter as SpeechRecognitionResult;if (vcResult!=null){string commandMode=vcResult.SemanticInterpretation.Properties["commandMode"][0];if(commandMode=="voice"){SpeakText(audioPlayer, String.Format("MSDN app heard you say {0}", vcResult.Text));HandleNlpCommand(vcResult);}else if (commandMode="text"){messageTextBox.Text=string.Format("working on your request \"{0}\"", vcResult.Text);HandleNlpCommand(vcResult);}}

phrase list :help narrow down the command

3.4 Example Using JavaScript

Command Monkey

website:   http://codefoster.com/commandMonkey

Cortana Voice Command  ----->   Phone -----------> Service ---------> Device

3.5 Localize the App for Relevant Languages

create different command sets

xml:lang="en-us"     

xml:lang="fr-fr"